Jul 22 02:20:41.150053 ip-10-0-134-165 systemd[1]: Starting Kubernetes Kubelet... Jul 22 02:20:41.643647 ip-10-0-134-165 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:20:41.643647 ip-10-0-134-165 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 02:20:41.643647 ip-10-0-134-165 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:20:41.643647 ip-10-0-134-165 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 02:20:41.643647 ip-10-0-134-165 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:20:41.644630 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.644507 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 02:20:41.649924 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649901 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:20:41.649924 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649918 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:20:41.649924 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649926 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:20:41.649924 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649929 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649934 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649938 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649942 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649946 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649949 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649953 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649958 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649965 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649968 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649973 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649977 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649982 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649985 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649989 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649993 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.649996 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650002 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650006 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650010 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:20:41.650170 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650014 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650018 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650021 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650028 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650033 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650038 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650042 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650046 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650051 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650055 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650060 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650064 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650068 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650072 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650078 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650082 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650086 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650090 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650094 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650099 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:20:41.651020 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650103 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650107 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650111 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650116 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650120 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650125 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650129 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650133 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650137 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650141 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650146 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650150 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650154 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650158 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650163 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650167 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650171 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650175 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650179 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650184 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:20:41.651693 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650187 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650191 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650195 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650200 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650204 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650208 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650212 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650217 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650222 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650226 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650231 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650235 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650239 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650243 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650247 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650251 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650260 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650265 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650270 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:20:41.652220 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650274 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650279 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650283 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650288 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650292 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650296 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650300 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650305 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650309 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650314 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650319 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650324 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650336 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650341 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650345 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650349 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.650353 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651569 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651582 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651587 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:20:41.652679 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651590 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651593 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651596 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651599 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651602 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651604 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651607 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651610 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651614 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651618 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651621 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651625 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651628 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651631 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651634 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651637 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651640 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651642 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651645 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:20:41.653218 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651648 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651650 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651653 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651655 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651658 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651661 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651663 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651666 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651668 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651671 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651674 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651685 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651688 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651691 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651694 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651697 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651699 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651702 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651705 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651709 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:20:41.653688 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651711 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651714 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651717 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651719 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651722 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651724 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651726 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651729 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651731 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651734 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651737 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651741 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651745 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651748 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651751 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651754 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651757 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651759 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651776 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:20:41.654235 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651780 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651783 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651785 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651794 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651798 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651801 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651805 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651809 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651812 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651814 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651817 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651820 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651822 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651825 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651827 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651830 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651832 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651835 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651837 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:20:41.654701 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651840 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651843 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651845 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651847 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651850 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651853 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651855 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651857 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651860 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651862 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651865 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651867 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651870 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651872 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651875 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651878 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651882 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651886 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.651888 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652288 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:20:41.655183 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652293 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652296 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652299 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652301 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652304 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652307 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652309 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652312 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652314 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652316 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652319 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652321 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652324 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652327 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652329 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652332 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652335 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652337 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652340 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652343 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:20:41.655652 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652345 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652348 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652352 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652355 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652359 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652362 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652365 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652368 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652372 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652376 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652378 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652382 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652385 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652387 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652389 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652393 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652395 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652398 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652400 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:20:41.656151 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652403 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652405 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652408 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652410 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652413 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652415 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652418 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652420 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652423 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652426 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652429 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652431 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652434 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652437 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652439 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652441 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652444 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652447 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652449 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652451 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:20:41.656621 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652454 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652457 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652461 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652464 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652466 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652469 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652472 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652474 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652477 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652479 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652482 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652484 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652487 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652489 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652492 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652494 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652497 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652500 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652502 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652505 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:20:41.657144 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652507 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652509 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652512 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652514 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652517 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652519 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652523 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652526 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652528 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652531 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652533 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652536 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652538 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652542 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652547 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652550 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652552 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652555 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.652558 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652628 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652635 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 02:20:41.657629 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652641 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652646 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652650 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652653 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652658 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652661 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652664 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652667 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652671 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652674 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652677 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652680 2561 flags.go:64] FLAG: --cgroup-root="" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652683 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652686 2561 flags.go:64] FLAG: --client-ca-file="" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652689 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652692 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652696 2561 flags.go:64] FLAG: --cluster-domain="" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652699 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652702 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652705 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652708 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652711 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652714 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 02:20:41.658164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652717 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652721 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652725 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652728 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652731 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652734 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652736 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652744 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652748 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652751 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652753 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652756 2561 flags.go:64] FLAG: --enable-server="true" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652759 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652777 2561 flags.go:64] FLAG: --event-burst="100" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652780 2561 flags.go:64] FLAG: --event-qps="50" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652783 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652786 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652789 2561 flags.go:64] FLAG: --eviction-hard="" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652793 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652795 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652798 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652801 2561 flags.go:64] FLAG: --eviction-soft="" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652804 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652806 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652809 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 02:20:41.658714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652812 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652815 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652818 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652820 2561 flags.go:64] FLAG: --feature-gates="" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652824 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652827 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652830 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652834 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652837 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652840 2561 flags.go:64] FLAG: --help="false" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652842 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-134-165.ec2.internal" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652845 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652848 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652851 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652855 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652859 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652861 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652864 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652867 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652870 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652873 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652876 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652879 2561 flags.go:64] FLAG: --kube-reserved="" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652881 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 02:20:41.659366 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652884 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652887 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652890 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652892 2561 flags.go:64] FLAG: --lock-file="" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652895 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652898 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652901 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652906 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652909 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652912 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652914 2561 flags.go:64] FLAG: --logging-format="text" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652918 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652921 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652924 2561 flags.go:64] FLAG: --manifest-url="" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652926 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652931 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652934 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652938 2561 flags.go:64] FLAG: --max-pods="110" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652941 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652943 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652946 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652949 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652952 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652955 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652959 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 02:20:41.659962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652965 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652969 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652971 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652974 2561 flags.go:64] FLAG: --pod-cidr="" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652977 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652983 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652985 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652988 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652991 2561 flags.go:64] FLAG: --port="10250" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652994 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.652997 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-007354726e9ac5791" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653000 2561 flags.go:64] FLAG: --qos-reserved="" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653003 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653006 2561 flags.go:64] FLAG: --register-node="true" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653008 2561 flags.go:64] FLAG: --register-with-taints="" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653012 2561 flags.go:64] FLAG: --registry-burst="10" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653015 2561 flags.go:64] FLAG: --registry-qps="5" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653018 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653020 2561 flags.go:64] FLAG: --reserved-memory="" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653023 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653026 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653029 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653032 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653034 2561 flags.go:64] FLAG: --runonce="false" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653037 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 02:20:41.660569 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653040 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653043 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653045 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653048 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653051 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653054 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653057 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653060 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653063 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653065 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653068 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653070 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653073 2561 flags.go:64] FLAG: --system-cgroups="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653076 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653081 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653084 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653087 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653090 2561 flags.go:64] FLAG: --tls-min-version="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653093 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653097 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653100 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653103 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653107 2561 flags.go:64] FLAG: --v="2" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653111 2561 flags.go:64] FLAG: --version="false" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653115 2561 flags.go:64] FLAG: --vmodule="" Jul 22 02:20:41.661193 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653119 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.653122 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653248 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653252 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653256 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653259 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653261 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653265 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653268 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653270 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653273 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653276 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653279 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653282 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653284 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653287 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653290 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653292 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653295 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653297 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:20:41.661819 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653300 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653302 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653305 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653307 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653310 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653313 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653318 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653322 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653326 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653330 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653334 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653337 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653340 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653343 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653346 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653349 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653352 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653355 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653358 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:20:41.662310 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653361 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653363 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653365 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653368 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653370 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653373 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653375 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653378 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653380 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653389 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653392 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653395 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653397 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653399 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653402 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653404 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653407 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653409 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653411 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653415 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:20:41.662858 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653418 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653421 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653424 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653427 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653429 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653432 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653434 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653437 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653439 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653442 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653444 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653447 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653449 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653452 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653454 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653457 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653459 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653462 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653464 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653467 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:20:41.663361 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653469 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653472 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653475 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653477 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653480 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653482 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653485 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653487 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653490 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653492 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653495 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653499 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653501 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653504 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653507 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653510 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653512 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653515 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653517 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653520 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:20:41.663857 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653522 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.653525 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.654155 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.660794 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.660813 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.660832 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.660842 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660897 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660903 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660906 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660909 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660912 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660915 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660918 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660921 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:20:41.664395 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660924 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660927 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660930 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660932 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660934 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660937 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660940 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660942 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660945 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660948 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660950 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660953 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660955 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660957 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660960 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660962 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660965 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660968 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660971 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660974 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:20:41.664775 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660978 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660982 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660985 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660989 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660991 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660994 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660996 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.660999 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661003 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661006 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661009 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661012 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661015 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661017 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661020 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661022 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661026 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661029 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661032 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:20:41.665250 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661035 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661037 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661040 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661043 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661046 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661049 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661051 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661054 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661057 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661060 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661063 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661066 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661069 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661071 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661074 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661077 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661080 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661083 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661085 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:20:41.665708 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661087 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661090 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661093 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661095 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661098 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661100 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661103 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661106 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661108 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661110 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661113 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661115 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661117 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661120 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661122 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661125 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661127 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661129 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661132 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661134 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:20:41.666278 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661137 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661139 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661141 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661144 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661147 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661150 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661152 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661156 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661158 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661161 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661164 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661166 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661168 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.661174 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661292 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:20:41.666870 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661296 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661299 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661302 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661304 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661307 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661310 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661312 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661315 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661317 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661320 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661322 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661325 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661327 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661330 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661332 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661335 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661337 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661340 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661342 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:20:41.667487 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661344 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661347 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661351 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661354 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661357 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661359 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661362 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661364 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661367 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661370 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661372 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661375 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661379 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661382 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661385 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661387 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661390 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661392 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661395 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661397 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:20:41.668267 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661399 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661402 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661405 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661407 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661410 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661412 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661415 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661417 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661419 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661422 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661424 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661427 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661429 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661433 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661437 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661440 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661442 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661445 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661447 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661450 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:20:41.668955 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661452 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661455 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661458 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661460 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661462 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661464 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661468 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661471 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661475 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661477 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661480 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661482 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661485 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661487 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661490 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661492 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661495 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661497 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661500 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:20:41.669457 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661502 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661504 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661507 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661509 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661512 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661514 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661516 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661519 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661522 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661524 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661527 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661529 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661532 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661534 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661537 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661540 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661542 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661544 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661547 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:20:41.669940 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:41.661549 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:20:41.670434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.661554 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:20:41.670434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.661664 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 02:20:41.670434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.663854 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 02:20:41.670434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.665126 2561 server.go:1019] "Starting client certificate rotation" Jul 22 02:20:41.670434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.665229 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:20:41.670434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.667071 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:20:41.670837 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.670783 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:20:41.672348 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.672329 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:20:41.682067 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.682050 2561 log.go:25] "Validated CRI v1 runtime API" Jul 22 02:20:41.686140 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.686124 2561 log.go:25] "Validated CRI v1 image API" Jul 22 02:20:41.687487 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.687469 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 02:20:41.690790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.690748 2561 fs.go:135] Filesystem UUIDs: map[4eb363af-94e8-43e4-94d9-0605c8290e89:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c4c7c151-ed4a-403d-8b5a-1bd845989952:/dev/nvme0n1p4] Jul 22 02:20:41.690869 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.690789 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 02:20:41.696348 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.696242 2561 manager.go:216] Machine: {Timestamp:2026-07-22 02:20:41.695736257 +0000 UTC m=+0.331473052 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099309 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e08f1a5acd9898dee020483139458 SystemUUID:ec2e08f1-a5ac-d989-8dee-020483139458 BootID:04ac7aaa-9fcc-49b2-821e-92940fc784a9 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:57:fb:71:8c:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:57:fb:71:8c:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:7d:3f:5b:3e:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 02:20:41.696348 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.696342 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 02:20:41.696464 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.696420 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 02:20:41.699539 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.699514 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 02:20:41.699684 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.699546 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 02:20:41.700741 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.700730 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 02:20:41.700787 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.700744 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 02:20:41.700787 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.700752 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:20:41.701000 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.700986 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:20:41.701504 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.701484 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:20:41.702794 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.702783 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 02:20:41.705551 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705540 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:20:41.705696 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705686 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 02:20:41.705724 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705718 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 22 02:20:41.705750 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705729 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 02:20:41.705793 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705753 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 02:20:41.705793 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705776 2561 kubelet.go:398] "Adding apiserver pod source" Jul 22 02:20:41.705793 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.705784 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 02:20:41.706159 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.706147 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:20:41.706229 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.706164 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:20:41.709408 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.709388 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 02:20:41.713056 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.713024 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 02:20:41.713142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.713080 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 02:20:41.714198 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714175 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 02:20:41.714198 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714196 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 02:20:41.714198 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714202 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714208 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714219 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714227 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714237 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714242 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714248 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714254 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714263 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 02:20:41.714383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.714268 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 02:20:41.715841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.715828 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 02:20:41.715841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.715840 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 02:20:41.717460 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.717437 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 02:20:41.717519 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.717450 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 02:20:41.719080 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.719066 2561 server.go:1268] "Started kubelet" Jul 22 02:20:41.719190 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.719161 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 02:20:41.719245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.719173 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 02:20:41.719367 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.719338 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 02:20:41.720043 ip-10-0-134-165 systemd[1]: Started Kubernetes Kubelet. Jul 22 02:20:41.720931 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.720916 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 02:20:41.721126 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.721012 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 22 02:20:41.723636 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.723610 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 02:20:41.726434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.726414 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 02:20:41.729200 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.726473 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-165.ec2.internal.18c47b3a13fa12f5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-165.ec2.internal,UID:ip-10-0-134-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-165.ec2.internal,},FirstTimestamp:2026-07-22 02:20:41.719034613 +0000 UTC m=+0.354771408,LastTimestamp:2026-07-22 02:20:41.719034613 +0000 UTC m=+0.354771408,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-165.ec2.internal,}" Jul 22 02:20:41.729345 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.729329 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 02:20:41.729414 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.729340 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 02:20:41.729414 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.729410 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:41.730026 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.729922 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 02:20:41.730026 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.729925 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 02:20:41.730026 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.729948 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 02:20:41.730244 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730143 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 22 02:20:41.730244 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730152 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 22 02:20:41.730244 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730238 2561 factory.go:55] Registering systemd factory Jul 22 02:20:41.730399 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730259 2561 factory.go:223] Registration of the systemd container factory successfully Jul 22 02:20:41.730571 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730554 2561 factory.go:153] Registering CRI-O factory Jul 22 02:20:41.730571 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730572 2561 factory.go:223] Registration of the crio container factory successfully Jul 22 02:20:41.730701 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730615 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 02:20:41.730701 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730638 2561 factory.go:103] Registering Raw factory Jul 22 02:20:41.730701 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.730652 2561 manager.go:1196] Started watching for new ooms in manager Jul 22 02:20:41.731186 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.731173 2561 manager.go:349] Starting recovery of all containers Jul 22 02:20:41.732701 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.732677 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 02:20:41.732701 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.732692 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 02:20:41.745329 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.745310 2561 manager.go:354] Recovery completed Jul 22 02:20:41.746963 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.746944 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wkhv" Jul 22 02:20:41.749466 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.749452 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.751251 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751236 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.751312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751272 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.751312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751285 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.751918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751905 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 02:20:41.751918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751917 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 02:20:41.752005 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751933 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:20:41.752005 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.751975 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.752695 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.752673 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.752804 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.752705 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.752804 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.752716 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.753113 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.753097 2561 policy_none.go:49] "None policy: Start" Jul 22 02:20:41.753179 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.753119 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 02:20:41.753179 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.753134 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 02:20:41.753487 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.753404 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-165.ec2.internal.18c47b3a15e5b2ac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-165.ec2.internal,UID:ip-10-0-134-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-165.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-165.ec2.internal,},FirstTimestamp:2026-07-22 02:20:41.751253676 +0000 UTC m=+0.386990480,LastTimestamp:2026-07-22 02:20:41.751253676 +0000 UTC m=+0.386990480,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-165.ec2.internal,}" Jul 22 02:20:41.754175 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.754162 2561 policy_none.go:47] "Start" Jul 22 02:20:41.754899 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.754877 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wkhv" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.796369 2561 manager.go:337] "Starting Device Plugin manager" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.796566 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.796581 2561 server.go:87] "Starting device plugin registration server" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.796825 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.796836 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.796919 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.797005 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.797014 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.797674 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 02:20:41.808090 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.797734 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:41.863182 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.863141 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 02:20:41.864535 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.864514 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 02:20:41.864604 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.864559 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 02:20:41.864604 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.864585 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 02:20:41.864689 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.864631 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 02:20:41.866878 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.866849 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:20:41.898006 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.897917 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.899780 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.899739 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.899920 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.899794 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.899920 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.899811 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.899920 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.899843 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-165.ec2.internal" Jul 22 02:20:41.909452 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.909430 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-165.ec2.internal" Jul 22 02:20:41.909559 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.909455 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-165.ec2.internal\": node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:41.937176 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.937146 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:41.964964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.964905 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal"] Jul 22 02:20:41.965113 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.965015 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.966136 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.966117 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.966136 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.966148 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.966279 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.966158 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.968218 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.968205 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.968324 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.968310 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:41.968364 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.968339 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.969100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.969081 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.969210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.969108 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.969210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.969118 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.969210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.969158 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.969210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.969185 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.969210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.969196 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.971099 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.971084 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" Jul 22 02:20:41.971178 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.971108 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:20:41.971940 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.971925 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:20:41.972010 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.971954 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:20:41.972010 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:41.971965 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:20:41.992479 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.992454 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-165.ec2.internal\" not found" node="ip-10-0-134-165.ec2.internal" Jul 22 02:20:41.996796 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:41.996780 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-165.ec2.internal\" not found" node="ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.030985 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.030960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7880506548359367b66d0c274b80e32c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal\" (UID: \"7880506548359367b66d0c274b80e32c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.031128 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.030994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2ed18d9f9ede21e451f5e83ef12c968d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-165.ec2.internal\" (UID: \"2ed18d9f9ede21e451f5e83ef12c968d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.031128 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.031018 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7880506548359367b66d0c274b80e32c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal\" (UID: \"7880506548359367b66d0c274b80e32c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.038064 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.038043 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.131650 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.131620 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7880506548359367b66d0c274b80e32c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal\" (UID: \"7880506548359367b66d0c274b80e32c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.131837 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.131657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7880506548359367b66d0c274b80e32c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal\" (UID: \"7880506548359367b66d0c274b80e32c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.131837 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.131698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2ed18d9f9ede21e451f5e83ef12c968d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-165.ec2.internal\" (UID: \"2ed18d9f9ede21e451f5e83ef12c968d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.131837 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.131716 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7880506548359367b66d0c274b80e32c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal\" (UID: \"7880506548359367b66d0c274b80e32c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.131837 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.131729 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7880506548359367b66d0c274b80e32c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal\" (UID: \"7880506548359367b66d0c274b80e32c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.131837 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.131751 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2ed18d9f9ede21e451f5e83ef12c968d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-165.ec2.internal\" (UID: \"2ed18d9f9ede21e451f5e83ef12c968d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.138717 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.138702 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.239577 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.239512 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.295726 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.295691 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.300606 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.300349 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" Jul 22 02:20:42.340243 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.340205 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.440772 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.440732 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.541305 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.541273 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.615368 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.615095 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:20:42.641733 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.641694 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.664900 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.664855 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 02:20:42.665396 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.665057 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:20:42.665396 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.665047 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:20:42.742222 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.742195 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.758776 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.758742 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 02:15:42 +0000 UTC" deadline="2028-01-13 20:17:02.361648981 +0000 UTC" Jul 22 02:20:42.758868 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.758798 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12977h56m19.602871248s" Jul 22 02:20:42.842368 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.842293 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:42.860981 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:42.860964 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:20:42.943281 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:42.943242 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:43.042153 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:43.042121 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed18d9f9ede21e451f5e83ef12c968d.slice/crio-8eeaaf86ee17f2e2065bee76c551679fccf034ce1ea650debf05d9e00970f65f WatchSource:0}: Error finding container 8eeaaf86ee17f2e2065bee76c551679fccf034ce1ea650debf05d9e00970f65f: Status 404 returned error can't find the container with id 8eeaaf86ee17f2e2065bee76c551679fccf034ce1ea650debf05d9e00970f65f Jul 22 02:20:43.042977 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:43.042956 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7880506548359367b66d0c274b80e32c.slice/crio-1143a4d99c09898fed6cf254243c868e524aa34642f1eb06673769ebceeb5eda WatchSource:0}: Error finding container 1143a4d99c09898fed6cf254243c868e524aa34642f1eb06673769ebceeb5eda: Status 404 returned error can't find the container with id 1143a4d99c09898fed6cf254243c868e524aa34642f1eb06673769ebceeb5eda Jul 22 02:20:43.044074 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.044056 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:43.049398 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.049377 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:20:43.144916 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.144823 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:43.245655 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.245623 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-165.ec2.internal\" not found" Jul 22 02:20:43.261968 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.261942 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:20:43.329866 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.329818 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" Jul 22 02:20:43.345849 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.345826 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:20:43.346978 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.346958 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" Jul 22 02:20:43.358927 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.358907 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:20:43.632091 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.632061 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:20:43.706218 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.706191 2561 apiserver.go:52] "Watching apiserver" Jul 22 02:20:43.712855 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.712833 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 02:20:43.713287 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.713256 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal","openshift-image-registry/node-ca-z8z26","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal","openshift-network-diagnostics/network-check-target-rdd2m","openshift-ovn-kubernetes/ovnkube-node-gfnjk","kube-system/global-pull-secret-syncer-k6hvt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br","openshift-cluster-node-tuning-operator/tuned-9f87x","openshift-dns/node-resolver-564zk","openshift-multus/multus-additional-cni-plugins-fgnn4","openshift-multus/multus-zl86t","openshift-multus/network-metrics-daemon-n9wv2","openshift-network-operator/iptables-alerter-jrnb2","kube-system/konnectivity-agent-wmhdc"] Jul 22 02:20:43.715807 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.715787 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.718168 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.718144 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sbz24\"" Jul 22 02:20:43.718305 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.718286 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.718383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.718364 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.718664 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.718646 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.720893 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.720874 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:43.721008 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.720952 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:43.721244 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.721224 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.721327 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.721281 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8zhn6\"" Jul 22 02:20:43.721483 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.721463 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.721581 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.721496 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 02:20:43.723659 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.723140 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.725082 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725064 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.725359 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725342 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.725439 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725354 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 02:20:43.725742 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725508 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kxwh5\"" Jul 22 02:20:43.725742 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725573 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 02:20:43.725742 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725736 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 02:20:43.725951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.725839 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.727992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.727530 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:43.727992 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.727609 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:43.727992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.727702 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.727992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.727532 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6qlxn\"" Jul 22 02:20:43.727992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.727955 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.729956 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.729938 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.730430 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.730413 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 02:20:43.732530 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.732506 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.732609 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.732595 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:43.733052 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.733031 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.733324 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.733300 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 02:20:43.733574 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.733560 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.733913 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.733896 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cn58s\"" Jul 22 02:20:43.736210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.736189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.738749 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.738731 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 02:20:43.739064 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739048 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.739273 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739248 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 02:20:43.739458 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739443 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k7v8w\"" Jul 22 02:20:43.739653 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739637 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739835 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739866 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99ffa421-75ab-4de2-af99-41a128fabc39-iptables-alerter-script\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739920 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739926 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbpj2\" (UniqueName: \"kubernetes.io/projected/66dde4a9-a236-4bda-a7e2-2da7d71d668d-kube-api-access-qbpj2\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.739976 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/820aa193-ad08-4733-86d5-50b1a391bbcb-tmp\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740027 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-registration-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740049 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-device-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysconfig\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740097 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-systemd\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740177 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-host\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740292 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740322 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-cnibin\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-cni-binary-copy\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740376 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysctl-d\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740389 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysctl-conf\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-sys\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740431 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99ffa421-75ab-4de2-af99-41a128fabc39-host-slash\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740443 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-run\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740457 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-etc-selinux\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740472 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wjzr\" (UniqueName: \"kubernetes.io/projected/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-kube-api-access-8wjzr\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740493 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66dde4a9-a236-4bda-a7e2-2da7d71d668d-hosts-file\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e149137e-7472-41b4-85a0-99c095e9db20-kubelet-config\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740535 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e149137e-7472-41b4-85a0-99c095e9db20-dbus\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-modprobe-d\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740577 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-tuned\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740600 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv44c\" (UniqueName: \"kubernetes.io/projected/820aa193-ad08-4733-86d5-50b1a391bbcb-kube-api-access-nv44c\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740623 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-socket-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740650 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxv4r\" (UniqueName: \"kubernetes.io/projected/61382fc5-f560-4435-a9f0-d0dc961dbe30-kube-api-access-hxv4r\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-var-lib-kubelet\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740694 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-sys-fs\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.740976 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6g62\" (UniqueName: \"kubernetes.io/projected/9e1c8b7f-c078-4409-8112-e34cf3420487-kube-api-access-t6g62\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740740 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-system-cni-dir\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-os-release\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrxkx\" (UniqueName: \"kubernetes.io/projected/99ffa421-75ab-4de2-af99-41a128fabc39-kube-api-access-zrxkx\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740834 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740859 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dde4a9-a236-4bda-a7e2-2da7d71d668d-tmp-dir\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740883 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740908 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-kubernetes\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.741603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.740931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-lib-modules\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.743411 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.743164 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:20:43.743411 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.743173 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.745342 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.745202 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 02:20:43.745342 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.745329 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-82xzn\"" Jul 22 02:20:43.745505 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.745434 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.747742 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.747725 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-phhzt\"" Jul 22 02:20:43.747860 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.747795 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 02:20:43.747860 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.747811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.747971 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.747878 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 02:20:43.748050 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.748028 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 02:20:43.750020 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.749911 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 02:20:43.750420 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.750390 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 02:20:43.750511 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.750433 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mlvgv\"" Jul 22 02:20:43.762548 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.762524 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6xhmz" Jul 22 02:20:43.770587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.770504 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6xhmz" Jul 22 02:20:43.831565 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.831535 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 02:20:43.842083 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66dde4a9-a236-4bda-a7e2-2da7d71d668d-hosts-file\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.842245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842089 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e149137e-7472-41b4-85a0-99c095e9db20-dbus\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.842245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842116 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-socket-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.842245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-daemon-config\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.842245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842177 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.842245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f29107fe-f689-4b7d-9b0b-39375612a435-host\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.842245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842227 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-sys-fs\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842250 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6g62\" (UniqueName: \"kubernetes.io/projected/9e1c8b7f-c078-4409-8112-e34cf3420487-kube-api-access-t6g62\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-cnibin\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842283 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-k8s-cni-cncf-io\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842284 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66dde4a9-a236-4bda-a7e2-2da7d71d668d-hosts-file\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-node-log\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-os-release\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxkx\" (UniqueName: \"kubernetes.io/projected/99ffa421-75ab-4de2-af99-41a128fabc39-kube-api-access-zrxkx\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842367 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-cni-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842369 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e149137e-7472-41b4-85a0-99c095e9db20-dbus\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842377 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-sys-fs\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-multus-certs\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-var-lib-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-ovn\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842469 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-os-release\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-socket-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dde4a9-a236-4bda-a7e2-2da7d71d668d-tmp-dir\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.842583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842546 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-kubernetes\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-lib-modules\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842630 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99ffa421-75ab-4de2-af99-41a128fabc39-iptables-alerter-script\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbpj2\" (UniqueName: \"kubernetes.io/projected/66dde4a9-a236-4bda-a7e2-2da7d71d668d-kube-api-access-qbpj2\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842682 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-registration-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-device-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842745 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-cni-binary-copy\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842775 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-kubelet\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842795 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-log-socket\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842794 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dde4a9-a236-4bda-a7e2-2da7d71d668d-tmp-dir\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842910 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-device-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.842919 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-kubernetes\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843033 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-lib-modules\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843082 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-registration-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8jw4\" (UniqueName: \"kubernetes.io/projected/b2306067-a195-47ee-83de-361228424831-kube-api-access-d8jw4\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.843614 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-systemd\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843558 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-host\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.843662 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.843666 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843693 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-systemd\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.843737 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:20:44.343713199 +0000 UTC m=+2.979450003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843740 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-host\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-conf-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.843776 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:44.343754894 +0000 UTC m=+2.979491694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843854 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-etc-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843891 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-cnibin\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843935 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-cnibin\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843952 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysctl-d\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843923 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99ffa421-75ab-4de2-af99-41a128fabc39-iptables-alerter-script\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.843982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysctl-conf\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.844482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-run\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844097 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysctl-d\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844105 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e149137e-7472-41b4-85a0-99c095e9db20-kubelet-config\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844101 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844141 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysctl-conf\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844140 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-run\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844071 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-modprobe-d\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844179 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e149137e-7472-41b4-85a0-99c095e9db20-kubelet-config\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-tuned\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844200 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-modprobe-d\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844215 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv44c\" (UniqueName: \"kubernetes.io/projected/820aa193-ad08-4733-86d5-50b1a391bbcb-kube-api-access-nv44c\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844235 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-os-release\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-socket-dir-parent\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844281 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdzn6\" (UniqueName: \"kubernetes.io/projected/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-kube-api-access-vdzn6\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.845203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844307 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-cni-bin\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844336 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxv4r\" (UniqueName: \"kubernetes.io/projected/61382fc5-f560-4435-a9f0-d0dc961dbe30-kube-api-access-hxv4r\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844361 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-var-lib-kubelet\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844385 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-kubelet\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844444 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-var-lib-kubelet\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844526 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844555 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-systemd-units\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844575 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-ovnkube-script-lib\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.844980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-system-cni-dir\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845045 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72cca779-c7b3-4615-9133-3aa76401f137-agent-certs\") pod \"konnectivity-agent-wmhdc\" (UID: \"72cca779-c7b3-4615-9133-3aa76401f137\") " pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-cni-bin\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845081 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61382fc5-f560-4435-a9f0-d0dc961dbe30-system-cni-dir\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-run-netns\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845195 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-run-ovn-kubernetes\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845322 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-netns\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-hostroot\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.846072 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/820aa193-ad08-4733-86d5-50b1a391bbcb-tmp\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-cni-multus\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845708 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-slash\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-systemd\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-ovnkube-config\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845821 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kcvf\" (UniqueName: \"kubernetes.io/projected/f29107fe-f689-4b7d-9b0b-39375612a435-kube-api-access-4kcvf\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysconfig\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845856 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-env-overrides\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845961 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2306067-a195-47ee-83de-361228424831-ovn-node-metrics-cert\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845966 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.845997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72cca779-c7b3-4615-9133-3aa76401f137-konnectivity-ca\") pod \"konnectivity-agent-wmhdc\" (UID: \"72cca779-c7b3-4615-9133-3aa76401f137\") " pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846031 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-sysconfig\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846056 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-cni-binary-copy\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-sys\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846156 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99ffa421-75ab-4de2-af99-41a128fabc39-host-slash\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.846881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-etc-kubernetes\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846220 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/820aa193-ad08-4733-86d5-50b1a391bbcb-sys\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846225 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-cni-netd\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846264 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f29107fe-f689-4b7d-9b0b-39375612a435-serviceca\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846287 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99ffa421-75ab-4de2-af99-41a128fabc39-host-slash\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-etc-selinux\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846334 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wjzr\" (UniqueName: \"kubernetes.io/projected/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-kube-api-access-8wjzr\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846367 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-system-cni-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846409 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e1c8b7f-c078-4409-8112-e34cf3420487-etc-selinux\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.847703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.846533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61382fc5-f560-4435-a9f0-d0dc961dbe30-cni-binary-copy\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.850241 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.850210 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/820aa193-ad08-4733-86d5-50b1a391bbcb-tmp\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.851169 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.851145 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/820aa193-ad08-4733-86d5-50b1a391bbcb-etc-tuned\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.853700 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.853679 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:43.853700 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.853701 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:43.853925 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.853715 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:43.853925 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:43.853807 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:20:44.353772976 +0000 UTC m=+2.989509791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:43.856522 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.856499 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrxkx\" (UniqueName: \"kubernetes.io/projected/99ffa421-75ab-4de2-af99-41a128fabc39-kube-api-access-zrxkx\") pod \"iptables-alerter-jrnb2\" (UID: \"99ffa421-75ab-4de2-af99-41a128fabc39\") " pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:43.856749 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.856543 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6g62\" (UniqueName: \"kubernetes.io/projected/9e1c8b7f-c078-4409-8112-e34cf3420487-kube-api-access-t6g62\") pod \"aws-ebs-csi-driver-node-9w8br\" (UID: \"9e1c8b7f-c078-4409-8112-e34cf3420487\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:43.859649 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.859626 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wjzr\" (UniqueName: \"kubernetes.io/projected/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-kube-api-access-8wjzr\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:43.859977 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.859952 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxv4r\" (UniqueName: \"kubernetes.io/projected/61382fc5-f560-4435-a9f0-d0dc961dbe30-kube-api-access-hxv4r\") pod \"multus-additional-cni-plugins-fgnn4\" (UID: \"61382fc5-f560-4435-a9f0-d0dc961dbe30\") " pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:43.860099 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.860047 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbpj2\" (UniqueName: \"kubernetes.io/projected/66dde4a9-a236-4bda-a7e2-2da7d71d668d-kube-api-access-qbpj2\") pod \"node-resolver-564zk\" (UID: \"66dde4a9-a236-4bda-a7e2-2da7d71d668d\") " pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:43.861525 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.861475 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv44c\" (UniqueName: \"kubernetes.io/projected/820aa193-ad08-4733-86d5-50b1a391bbcb-kube-api-access-nv44c\") pod \"tuned-9f87x\" (UID: \"820aa193-ad08-4733-86d5-50b1a391bbcb\") " pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:43.870089 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.870043 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" event={"ID":"2ed18d9f9ede21e451f5e83ef12c968d","Type":"ContainerStarted","Data":"8eeaaf86ee17f2e2065bee76c551679fccf034ce1ea650debf05d9e00970f65f"} Jul 22 02:20:43.871472 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.871438 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" event={"ID":"7880506548359367b66d0c274b80e32c","Type":"ContainerStarted","Data":"1143a4d99c09898fed6cf254243c868e524aa34642f1eb06673769ebceeb5eda"} Jul 22 02:20:43.946851 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946763 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-etc-kubernetes\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.946851 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946804 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-cni-netd\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946877 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f29107fe-f689-4b7d-9b0b-39375612a435-serviceca\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946911 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-system-cni-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946917 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-cni-netd\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946879 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-etc-kubernetes\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946957 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-daemon-config\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.946986 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947012 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f29107fe-f689-4b7d-9b0b-39375612a435-host\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-cnibin\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947047 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947012 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-system-cni-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947067 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-k8s-cni-cncf-io\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947093 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-node-log\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947111 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947101 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f29107fe-f689-4b7d-9b0b-39375612a435-host\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-cni-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947133 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-cnibin\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947163 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-cni-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947175 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-k8s-cni-cncf-io\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947186 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-node-log\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947162 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-multus-certs\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947202 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-multus-certs\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-var-lib-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-ovn\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947301 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-var-lib-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947314 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-cni-binary-copy\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947343 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-kubelet\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947347 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-ovn\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947369 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-log-socket\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947390 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-kubelet\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947399 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8jw4\" (UniqueName: \"kubernetes.io/projected/b2306067-a195-47ee-83de-361228424831-kube-api-access-d8jw4\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-conf-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.947790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f29107fe-f689-4b7d-9b0b-39375612a435-serviceca\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-etc-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947434 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-log-socket\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-conf-dir\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-os-release\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-socket-dir-parent\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947602 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-daemon-config\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947612 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947604 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdzn6\" (UniqueName: \"kubernetes.io/projected/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-kube-api-access-vdzn6\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-cni-bin\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947681 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-os-release\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947684 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-kubelet\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947724 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-multus-socket-dir-parent\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947724 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-cni-bin\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947730 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-kubelet\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947576 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-etc-openvswitch\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947724 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-cni-binary-copy\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.948770 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947744 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-systemd-units\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947770 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-systemd-units\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947780 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-ovnkube-script-lib\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72cca779-c7b3-4615-9133-3aa76401f137-agent-certs\") pod \"konnectivity-agent-wmhdc\" (UID: \"72cca779-c7b3-4615-9133-3aa76401f137\") " pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-cni-bin\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-run-netns\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947925 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-run-ovn-kubernetes\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947920 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-cni-bin\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-netns\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947959 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-run-netns\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-hostroot\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.947989 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-run-ovn-kubernetes\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-cni-multus\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948012 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-run-netns\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948038 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-hostroot\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-slash\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948059 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-host-var-lib-cni-multus\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948071 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-systemd\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.949425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948099 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-host-slash\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948108 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2306067-a195-47ee-83de-361228424831-run-systemd\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-ovnkube-config\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948156 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kcvf\" (UniqueName: \"kubernetes.io/projected/f29107fe-f689-4b7d-9b0b-39375612a435-kube-api-access-4kcvf\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948184 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-env-overrides\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948212 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2306067-a195-47ee-83de-361228424831-ovn-node-metrics-cert\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948237 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72cca779-c7b3-4615-9133-3aa76401f137-konnectivity-ca\") pod \"konnectivity-agent-wmhdc\" (UID: \"72cca779-c7b3-4615-9133-3aa76401f137\") " pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948460 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-ovnkube-script-lib\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948468 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-ovnkube-config\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948765 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2306067-a195-47ee-83de-361228424831-env-overrides\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.950100 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.948857 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72cca779-c7b3-4615-9133-3aa76401f137-konnectivity-ca\") pod \"konnectivity-agent-wmhdc\" (UID: \"72cca779-c7b3-4615-9133-3aa76401f137\") " pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.952299 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.952249 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72cca779-c7b3-4615-9133-3aa76401f137-agent-certs\") pod \"konnectivity-agent-wmhdc\" (UID: \"72cca779-c7b3-4615-9133-3aa76401f137\") " pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:43.952874 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.952523 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2306067-a195-47ee-83de-361228424831-ovn-node-metrics-cert\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.960929 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.960876 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kcvf\" (UniqueName: \"kubernetes.io/projected/f29107fe-f689-4b7d-9b0b-39375612a435-kube-api-access-4kcvf\") pod \"node-ca-z8z26\" (UID: \"f29107fe-f689-4b7d-9b0b-39375612a435\") " pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:43.961972 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.961946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8jw4\" (UniqueName: \"kubernetes.io/projected/b2306067-a195-47ee-83de-361228424831-kube-api-access-d8jw4\") pod \"ovnkube-node-gfnjk\" (UID: \"b2306067-a195-47ee-83de-361228424831\") " pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:43.963192 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:43.963160 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdzn6\" (UniqueName: \"kubernetes.io/projected/e80114d7-e030-4d8f-98bb-f99f3d6d51b8-kube-api-access-vdzn6\") pod \"multus-zl86t\" (UID: \"e80114d7-e030-4d8f-98bb-f99f3d6d51b8\") " pod="openshift-multus/multus-zl86t" Jul 22 02:20:44.027080 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.026825 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9f87x" Jul 22 02:20:44.035469 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.035448 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jrnb2" Jul 22 02:20:44.039425 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:44.039388 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820aa193_ad08_4733_86d5_50b1a391bbcb.slice/crio-8068cbdba4c221742a44a4725af577583c4acc1223648780cd67bc9dbc68bf4c WatchSource:0}: Error finding container 8068cbdba4c221742a44a4725af577583c4acc1223648780cd67bc9dbc68bf4c: Status 404 returned error can't find the container with id 8068cbdba4c221742a44a4725af577583c4acc1223648780cd67bc9dbc68bf4c Jul 22 02:20:44.046351 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.046323 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" Jul 22 02:20:44.054374 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.054202 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-564zk" Jul 22 02:20:44.058028 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:44.058003 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ffa421_75ab_4de2_af99_41a128fabc39.slice/crio-354cc85a7e535eee60f1b62696f9dff87a1eeab088c5fc87b2384a024d7fe9cb WatchSource:0}: Error finding container 354cc85a7e535eee60f1b62696f9dff87a1eeab088c5fc87b2384a024d7fe9cb: Status 404 returned error can't find the container with id 354cc85a7e535eee60f1b62696f9dff87a1eeab088c5fc87b2384a024d7fe9cb Jul 22 02:20:44.060338 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.060317 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" Jul 22 02:20:44.065273 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:44.065167 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61382fc5_f560_4435_a9f0_d0dc961dbe30.slice/crio-12b0b055033edc5ef78ca3e16f4ed3b2cb4d4e8c6b368149d164e7f69b6d9499 WatchSource:0}: Error finding container 12b0b055033edc5ef78ca3e16f4ed3b2cb4d4e8c6b368149d164e7f69b6d9499: Status 404 returned error can't find the container with id 12b0b055033edc5ef78ca3e16f4ed3b2cb4d4e8c6b368149d164e7f69b6d9499 Jul 22 02:20:44.067401 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.067112 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:20:44.073529 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:44.073431 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66dde4a9_a236_4bda_a7e2_2da7d71d668d.slice/crio-749d6881291eb09887d32176b64186d76183edb4f0a3fedd373c61381dab5070 WatchSource:0}: Error finding container 749d6881291eb09887d32176b64186d76183edb4f0a3fedd373c61381dab5070: Status 404 returned error can't find the container with id 749d6881291eb09887d32176b64186d76183edb4f0a3fedd373c61381dab5070 Jul 22 02:20:44.074285 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.074264 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zl86t" Jul 22 02:20:44.078558 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:44.078526 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1c8b7f_c078_4409_8112_e34cf3420487.slice/crio-46725175b123fd47a2bb7a40c638d31c96a2ae7dd909f50b6c2bda67877e0f08 WatchSource:0}: Error finding container 46725175b123fd47a2bb7a40c638d31c96a2ae7dd909f50b6c2bda67877e0f08: Status 404 returned error can't find the container with id 46725175b123fd47a2bb7a40c638d31c96a2ae7dd909f50b6c2bda67877e0f08 Jul 22 02:20:44.081489 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.081469 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z8z26" Jul 22 02:20:44.087385 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.087365 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:20:44.110315 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:20:44.110284 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29107fe_f689_4b7d_9b0b_39375612a435.slice/crio-1b3e58adff4e0c9642b18c2c7c374b0629af8d54cfab7a04bf019814ca899973 WatchSource:0}: Error finding container 1b3e58adff4e0c9642b18c2c7c374b0629af8d54cfab7a04bf019814ca899973: Status 404 returned error can't find the container with id 1b3e58adff4e0c9642b18c2c7c374b0629af8d54cfab7a04bf019814ca899973 Jul 22 02:20:44.350500 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.350317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:44.350500 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.350379 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:44.350500 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.350477 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:44.350763 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.350524 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:44.350763 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.350561 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:45.35053992 +0000 UTC m=+3.986276721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:44.350763 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.350587 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:20:45.350569697 +0000 UTC m=+3.986306485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:44.451554 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.451508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:44.451763 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.451739 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:44.451857 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.451774 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:44.451857 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.451810 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:44.451976 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.451881 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:20:45.451857253 +0000 UTC m=+4.087594055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:44.774285 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.774185 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:15:44 +0000 UTC" deadline="2028-02-22 02:56:42.571802091 +0000 UTC" Jul 22 02:20:44.774285 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.774224 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13920h35m57.797582322s" Jul 22 02:20:44.865501 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.865466 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:44.865693 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.865609 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:44.865771 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.865698 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:44.865832 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:44.865773 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:44.875756 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.875663 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wmhdc" event={"ID":"72cca779-c7b3-4615-9133-3aa76401f137","Type":"ContainerStarted","Data":"6a4348a2c08774bef87ed376c438192687f7635ceba181a8402368254bd5fc6a"} Jul 22 02:20:44.881407 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.881018 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zl86t" event={"ID":"e80114d7-e030-4d8f-98bb-f99f3d6d51b8","Type":"ContainerStarted","Data":"8d586f5048cdfc8ddd2193084d6e1dcbd264659f4cdf06f181ef933f7f92b407"} Jul 22 02:20:44.882842 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.882773 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"09294d749640389e675671682a1d494f149ab4138bf7da3c595422861cf77c17"} Jul 22 02:20:44.891766 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.889174 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" event={"ID":"9e1c8b7f-c078-4409-8112-e34cf3420487","Type":"ContainerStarted","Data":"46725175b123fd47a2bb7a40c638d31c96a2ae7dd909f50b6c2bda67877e0f08"} Jul 22 02:20:44.901049 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.901010 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerStarted","Data":"12b0b055033edc5ef78ca3e16f4ed3b2cb4d4e8c6b368149d164e7f69b6d9499"} Jul 22 02:20:44.907186 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.907150 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jrnb2" event={"ID":"99ffa421-75ab-4de2-af99-41a128fabc39","Type":"ContainerStarted","Data":"354cc85a7e535eee60f1b62696f9dff87a1eeab088c5fc87b2384a024d7fe9cb"} Jul 22 02:20:44.909505 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.909478 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-564zk" event={"ID":"66dde4a9-a236-4bda-a7e2-2da7d71d668d","Type":"ContainerStarted","Data":"749d6881291eb09887d32176b64186d76183edb4f0a3fedd373c61381dab5070"} Jul 22 02:20:44.910924 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.910831 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9f87x" event={"ID":"820aa193-ad08-4733-86d5-50b1a391bbcb","Type":"ContainerStarted","Data":"8068cbdba4c221742a44a4725af577583c4acc1223648780cd67bc9dbc68bf4c"} Jul 22 02:20:44.913435 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:44.913376 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z8z26" event={"ID":"f29107fe-f689-4b7d-9b0b-39375612a435","Type":"ContainerStarted","Data":"1b3e58adff4e0c9642b18c2c7c374b0629af8d54cfab7a04bf019814ca899973"} Jul 22 02:20:45.303751 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.303437 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:20:45.360883 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.360838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:45.361074 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.360962 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:45.361162 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.361147 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:45.361238 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.361222 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:47.361201422 +0000 UTC m=+5.996938212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:45.361325 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.361308 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:45.361379 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.361346 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:20:47.361332276 +0000 UTC m=+5.997069068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:45.462195 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.461976 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:45.462195 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.462195 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:45.462432 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.462214 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:45.462432 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.462227 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:45.462432 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.462286 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:20:47.46226741 +0000 UTC m=+6.098004215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:45.775609 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.775450 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:15:44 +0000 UTC" deadline="2028-03-14 15:58:10.688891612 +0000 UTC" Jul 22 02:20:45.775609 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.775522 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14437h37m24.91337472s" Jul 22 02:20:45.866334 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:45.865750 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:45.866334 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:45.865936 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:46.865624 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:46.865332 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:46.865624 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:46.865625 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:46.866250 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:46.865728 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:46.866250 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:46.866217 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:46.930148 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:46.930104 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" event={"ID":"2ed18d9f9ede21e451f5e83ef12c968d","Type":"ContainerStarted","Data":"3c577512ec1a8dc6fe86a9f2463b24e042119588c6c55086073068499bab3050"} Jul 22 02:20:47.377805 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.377716 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:47.377805 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.377773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:47.378071 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.377946 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:47.378071 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.378036 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:20:51.377986841 +0000 UTC m=+10.013723640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:47.378571 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.378550 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:47.378653 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.378614 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:51.378596278 +0000 UTC m=+10.014333078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:47.478175 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.478135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:47.478372 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.478338 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:47.478372 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.478362 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:47.478480 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.478374 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:47.478480 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.478475 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:20:51.47845404 +0000 UTC m=+10.114190849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:47.865864 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.865535 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:47.866353 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:47.865996 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:47.937444 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.937091 2561 generic.go:356] "Generic (PLEG): container finished" podID="7880506548359367b66d0c274b80e32c" containerID="7c276c3f9bb4d5426a2187624b18ada67cabf8a91d8eb8281a2a19ee73cd20b2" exitCode=0 Jul 22 02:20:47.937444 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.937207 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" event={"ID":"7880506548359367b66d0c274b80e32c","Type":"ContainerDied","Data":"7c276c3f9bb4d5426a2187624b18ada67cabf8a91d8eb8281a2a19ee73cd20b2"} Jul 22 02:20:47.957053 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:47.955728 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-165.ec2.internal" podStartSLOduration=3.955705191 podStartE2EDuration="3.955705191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:46.947278589 +0000 UTC m=+5.583015400" watchObservedRunningTime="2026-07-22 02:20:47.955705191 +0000 UTC m=+6.591442002" Jul 22 02:20:48.865982 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:48.865227 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:48.865982 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:48.865362 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:48.865982 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:48.865429 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:48.865982 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:48.865479 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:49.865271 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:49.865234 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:49.865456 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:49.865396 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:50.865365 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:50.865326 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:50.865767 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:50.865374 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:50.865767 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:50.865466 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:50.865924 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:50.865900 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:51.411521 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:51.411471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:51.411738 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:51.411537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:51.411738 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.411707 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:51.411884 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.411769 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:20:59.411753148 +0000 UTC m=+18.047489953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:51.412250 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.412214 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:51.412394 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.412274 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:59.412256662 +0000 UTC m=+18.047993468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:51.512667 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:51.512625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:51.512889 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.512858 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:51.512889 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.512914 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:51.512889 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.512929 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:51.513203 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.512998 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:20:59.512977311 +0000 UTC m=+18.148714110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:51.866381 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:51.866341 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:51.866856 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:51.866485 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:52.865280 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:52.865250 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:52.865478 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:52.865393 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:52.865478 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:52.865446 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:52.865599 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:52.865580 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:53.865191 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:53.865153 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:53.865647 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:53.865315 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:54.865575 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:54.865537 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:54.866099 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:54.865684 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:54.866213 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:54.866196 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:54.866325 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:54.866304 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:55.864867 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:55.864838 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:55.865028 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:55.864954 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:56.865384 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:56.865345 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:56.865791 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:56.865483 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:56.865791 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:56.865530 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:56.865791 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:56.865630 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:57.864973 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:57.864938 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:57.865159 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:57.865082 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:20:58.865122 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:58.865083 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:58.865665 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:58.865083 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:58.865665 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:58.865203 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:20:58.865665 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:58.865291 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:20:59.479138 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:59.479098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:20:59.479138 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:59.479144 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:59.479420 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.479260 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:59.479420 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.479260 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:59.479420 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.479325 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:21:15.479308417 +0000 UTC m=+34.115045206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:20:59.479420 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.479344 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:15.479334753 +0000 UTC m=+34.115071541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:20:59.580369 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:59.580329 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:20:59.580552 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.580501 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:20:59.580552 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.580520 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:20:59.580552 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.580529 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:59.580697 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.580583 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:21:15.580566752 +0000 UTC m=+34.216303542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:20:59.865492 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:20:59.865461 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:20:59.865976 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:20:59.865618 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:00.865504 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:00.865469 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:00.865934 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:00.865482 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:00.865934 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:00.865606 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:00.865934 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:00.865686 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:01.866621 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:01.866581 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:01.867121 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:01.866732 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:02.865583 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:02.865540 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:02.865820 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:02.865541 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:02.865820 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:02.865676 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:02.865820 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:02.865809 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:03.865264 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:03.865225 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:03.865750 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:03.865375 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:04.865129 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.864912 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:04.865281 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:04.865250 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:04.865651 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.865092 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:04.865800 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:04.865754 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:04.974797 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.974465 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z8z26" event={"ID":"f29107fe-f689-4b7d-9b0b-39375612a435","Type":"ContainerStarted","Data":"ded4a0bfffcb01eabb9255a4a6cac2abedc356da6f2f49cd5bf6554ac5f3336f"} Jul 22 02:21:04.976198 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.976118 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wmhdc" event={"ID":"72cca779-c7b3-4615-9133-3aa76401f137","Type":"ContainerStarted","Data":"a1d57f18ccc673a0fb67bf4c4ac5df6126b48ffed244f7502ffadd704d9b714c"} Jul 22 02:21:04.978951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.978847 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zl86t" event={"ID":"e80114d7-e030-4d8f-98bb-f99f3d6d51b8","Type":"ContainerStarted","Data":"09baab15e0a09797ba599c3002c482624ed93c1b3dc4a7a4e66a86d486ff9053"} Jul 22 02:21:04.981655 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.981193 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"40a65c622b269a6fc5f622cf7173f0a106a56cd7002625b95dcee5a69a7f8bd5"} Jul 22 02:21:04.984090 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.983579 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" event={"ID":"9e1c8b7f-c078-4409-8112-e34cf3420487","Type":"ContainerStarted","Data":"805603d24ff3ec460b2e0ebb30891dfd7200f0fae701318c6f0f4149e1e5afd6"} Jul 22 02:21:04.985852 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.985320 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerStarted","Data":"917336e2b533966d1264bdc9914b8c02d2160548ec1d30413df30f2e05b3cf04"} Jul 22 02:21:04.987747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.987464 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-564zk" event={"ID":"66dde4a9-a236-4bda-a7e2-2da7d71d668d","Type":"ContainerStarted","Data":"754b569a78dc134fd3c810e42f16b0347a4adb35933ba2bd947dc31a8310b2c4"} Jul 22 02:21:04.989995 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.989594 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z8z26" podStartSLOduration=1.395797425 podStartE2EDuration="21.989575392s" totalImagesPullingTime="20.593777967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:04.988635072 +0000 UTC m=+23.624371877" watchObservedRunningTime="2026-07-22 02:21:04.989575392 +0000 UTC m=+23.625312197" Jul 22 02:21:04.990196 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.990164 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9f87x" event={"ID":"820aa193-ad08-4733-86d5-50b1a391bbcb","Type":"ContainerStarted","Data":"fb91f44c8b2ce2c646d804084ae936dc10bc9e76eace3d1793d860d69c27a06e"} Jul 22 02:21:04.993230 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:04.992942 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" event={"ID":"7880506548359367b66d0c274b80e32c","Type":"ContainerStarted","Data":"b9c6c849a92f2acf53e38a98a6e259c8a23d73e493decfb18fa80aaf01d77e15"} Jul 22 02:21:05.002725 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.002672 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wmhdc" podStartSLOduration=1.407118013 podStartE2EDuration="22.002652092s" totalImagesPullingTime="20.595534079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.001964143 +0000 UTC m=+23.637700955" watchObservedRunningTime="2026-07-22 02:21:05.002652092 +0000 UTC m=+23.638388947" Jul 22 02:21:05.062729 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.062687 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-564zk" podStartSLOduration=2.443891452 podStartE2EDuration="23.062669132s" totalImagesPullingTime="20.61877768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.041606582 +0000 UTC m=+23.677343386" watchObservedRunningTime="2026-07-22 02:21:05.062669132 +0000 UTC m=+23.698405934" Jul 22 02:21:05.062900 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.062847 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zl86t" podStartSLOduration=1.408032763 podStartE2EDuration="22.062839456s" totalImagesPullingTime="20.654806693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.062437502 +0000 UTC m=+23.698174308" watchObservedRunningTime="2026-07-22 02:21:05.062839456 +0000 UTC m=+23.698576260" Jul 22 02:21:05.076918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.076872 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-165.ec2.internal" podStartSLOduration=21.076852874 podStartE2EDuration="21.076852874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.076325638 +0000 UTC m=+23.712062445" watchObservedRunningTime="2026-07-22 02:21:05.076852874 +0000 UTC m=+23.712589678" Jul 22 02:21:05.093746 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.093709 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9f87x" podStartSLOduration=2.434514097 podStartE2EDuration="23.093694648s" totalImagesPullingTime="20.659180551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:05.093554722 +0000 UTC m=+23.729291528" watchObservedRunningTime="2026-07-22 02:21:05.093694648 +0000 UTC m=+23.729431453" Jul 22 02:21:05.865727 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.865691 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:05.866495 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:05.865857 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:05.998076 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.997993 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"90be6bfa477c323a9782f56f77f97c084c53cb5c330f46f553507ae4e99e1b1b"} Jul 22 02:21:05.998076 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.998068 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"c5b09a37c1aa87673882ecaabdbff6d79e6bdfac57fd909d1c413ee88bf99ed9"} Jul 22 02:21:05.998427 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.998109 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"803b9e0e986202ce03aaf72b6443ef8366f72c2ef1aa6989b44c0b423550166f"} Jul 22 02:21:05.998427 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.998147 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"c16224ffb6674a7b191b4aa1045a8780982a09f21ea273ed06e89b9138f75d15"} Jul 22 02:21:05.998427 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.998181 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"b1d39f3bf5792cff9c1613ef86182699fe5690d25080c060334b0d29d32ac840"} Jul 22 02:21:05.999384 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.999361 2561 generic.go:356] "Generic (PLEG): container finished" podID="61382fc5-f560-4435-a9f0-d0dc961dbe30" containerID="917336e2b533966d1264bdc9914b8c02d2160548ec1d30413df30f2e05b3cf04" exitCode=0 Jul 22 02:21:05.999490 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:05.999464 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerDied","Data":"917336e2b533966d1264bdc9914b8c02d2160548ec1d30413df30f2e05b3cf04"} Jul 22 02:21:06.116567 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.116530 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:21:06.117377 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.117360 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:21:06.287639 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.287615 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 02:21:06.814222 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.814133 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T02:21:06.287632186Z","UUID":"1f9f8191-78e1-4fd5-8922-3c18d7d13e73","Handler":null,"Name":"","Endpoint":""} Jul 22 02:21:06.815718 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.815697 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 02:21:06.815861 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.815733 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 02:21:06.865398 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.865373 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:06.865517 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:06.865373 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:06.865517 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:06.865488 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:06.865632 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:06.865553 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:07.008835 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:07.008800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" event={"ID":"9e1c8b7f-c078-4409-8112-e34cf3420487","Type":"ContainerStarted","Data":"9cdfb26ba23ca0587996edb1106b7c0f01b838bd3820ac027261b0d3604c5c58"} Jul 22 02:21:07.010571 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:07.010539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jrnb2" event={"ID":"99ffa421-75ab-4de2-af99-41a128fabc39","Type":"ContainerStarted","Data":"03a496aa411acf95e973466e09ed557d0b691f0d1ed1e45f05540dbf15f7c1d4"} Jul 22 02:21:07.188741 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:07.188657 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:21:07.189272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:07.189252 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wmhdc" Jul 22 02:21:07.205253 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:07.205210 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jrnb2" podStartSLOduration=4.556892858 podStartE2EDuration="25.205194096s" totalImagesPullingTime="20.648301238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:07.025392863 +0000 UTC m=+25.661129670" watchObservedRunningTime="2026-07-22 02:21:07.205194096 +0000 UTC m=+25.840930904" Jul 22 02:21:07.865708 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:07.865678 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:07.865919 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:07.865834 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:08.016600 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:08.016563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"340c43d4c632eada2769c3ff9bcf96fb47a381ef2579983f6081e8f37807da30"} Jul 22 02:21:08.866034 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:08.865805 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:08.866223 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:08.865805 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:08.866223 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:08.866150 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:08.866360 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:08.866221 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:09.021778 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:09.021726 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" event={"ID":"9e1c8b7f-c078-4409-8112-e34cf3420487","Type":"ContainerStarted","Data":"3c2e43ed525c8befa3851f3ef40b131c5a6648c84e0a9ede5ca6940864b4dc73"} Jul 22 02:21:09.039574 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:09.039525 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9w8br" podStartSLOduration=2.5874553970000003 podStartE2EDuration="26.039511231s" totalImagesPullingTime="23.452055834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:09.038525998 +0000 UTC m=+27.674262794" watchObservedRunningTime="2026-07-22 02:21:09.039511231 +0000 UTC m=+27.675248038" Jul 22 02:21:09.865440 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:09.865404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:09.865627 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:09.865556 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:10.865439 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:10.865414 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:10.865964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:10.865414 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:10.865964 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:10.865510 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:10.865964 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:10.865584 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:11.030931 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.030748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" event={"ID":"b2306067-a195-47ee-83de-361228424831","Type":"ContainerStarted","Data":"6429e433e0c60aa678c3dcc4858f7443a7f003313559e3475e3a371d8c273310"} Jul 22 02:21:11.031075 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.031012 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:21:11.031156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.031142 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:21:11.031209 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.031199 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:21:11.047917 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.047888 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:21:11.048241 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.048226 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:21:11.059184 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.059132 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" podStartSLOduration=7.377292329 podStartE2EDuration="28.059118875s" totalImagesPullingTime="20.681826546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:11.058638508 +0000 UTC m=+29.694375316" watchObservedRunningTime="2026-07-22 02:21:11.059118875 +0000 UTC m=+29.694855679" Jul 22 02:21:11.865468 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:11.865433 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:11.865968 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:11.865527 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:12.034412 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:12.034375 2561 generic.go:356] "Generic (PLEG): container finished" podID="61382fc5-f560-4435-a9f0-d0dc961dbe30" containerID="d339c1596380dbd42d265c5254f996a03acdf5083a9039e240c2c03274911469" exitCode=0 Jul 22 02:21:12.034599 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:12.034459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerDied","Data":"d339c1596380dbd42d265c5254f996a03acdf5083a9039e240c2c03274911469"} Jul 22 02:21:12.865211 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:12.865184 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:12.865411 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:12.865285 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:12.865411 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:12.865323 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:12.865501 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:12.865464 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:13.109138 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:13.109105 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k6hvt"] Jul 22 02:21:13.109294 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:13.109263 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:13.109396 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:13.109376 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:13.111791 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:13.111752 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n9wv2"] Jul 22 02:21:13.111903 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:13.111863 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:13.111963 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:13.111942 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:13.115118 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:13.115099 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rdd2m"] Jul 22 02:21:13.115196 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:13.115181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:13.115285 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:13.115246 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:14.040623 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:14.040594 2561 generic.go:356] "Generic (PLEG): container finished" podID="61382fc5-f560-4435-a9f0-d0dc961dbe30" containerID="8f3770a8c5d39d6f4feb1e2624a71c2fcb1296a7449ac3bacc1d3cad8b2ee13c" exitCode=0 Jul 22 02:21:14.041002 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:14.040632 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerDied","Data":"8f3770a8c5d39d6f4feb1e2624a71c2fcb1296a7449ac3bacc1d3cad8b2ee13c"} Jul 22 02:21:14.865949 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:14.865690 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:14.866109 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:14.865690 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:14.866109 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:14.865992 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:14.866109 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:14.865707 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:14.866109 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:14.866082 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:14.866273 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:14.866123 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:15.045384 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:15.045353 2561 generic.go:356] "Generic (PLEG): container finished" podID="61382fc5-f560-4435-a9f0-d0dc961dbe30" containerID="70d04bc79658e59d6e896262753faef9041f488bbf3c1529e5c64ea97fd3fa5e" exitCode=0 Jul 22 02:21:15.045725 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:15.045397 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerDied","Data":"70d04bc79658e59d6e896262753faef9041f488bbf3c1529e5c64ea97fd3fa5e"} Jul 22 02:21:15.493451 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:15.493416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:15.493451 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:15.493456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:15.493692 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.493573 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:21:15.493692 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.493582 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:21:15.493692 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.493631 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs podName:b3c2f13c-6b3c-4292-9b7d-8d184363dccb nodeName:}" failed. No retries permitted until 2026-07-22 02:21:47.493613832 +0000 UTC m=+66.129350616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs") pod "network-metrics-daemon-n9wv2" (UID: "b3c2f13c-6b3c-4292-9b7d-8d184363dccb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:21:15.493692 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.493653 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret podName:e149137e-7472-41b4-85a0-99c095e9db20 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:47.493641512 +0000 UTC m=+66.129378296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret") pod "global-pull-secret-syncer-k6hvt" (UID: "e149137e-7472-41b4-85a0-99c095e9db20") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:21:15.594342 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:15.594302 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:15.594520 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.594497 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:21:15.594520 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.594520 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:21:15.594616 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.594529 2561 projected.go:196] Error preparing data for projected volume kube-api-access-w7gzr for pod openshift-network-diagnostics/network-check-target-rdd2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:21:15.594616 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:15.594582 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr podName:3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad nodeName:}" failed. No retries permitted until 2026-07-22 02:21:47.594566042 +0000 UTC m=+66.230302827 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w7gzr" (UniqueName: "kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr") pod "network-check-target-rdd2m" (UID: "3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:21:16.865429 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:16.865394 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:16.865899 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:16.865394 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:16.865899 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:16.865528 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6hvt" podUID="e149137e-7472-41b4-85a0-99c095e9db20" Jul 22 02:21:16.865899 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:16.865627 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9wv2" podUID="b3c2f13c-6b3c-4292-9b7d-8d184363dccb" Jul 22 02:21:16.865899 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:16.865394 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:16.865899 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:16.865697 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdd2m" podUID="3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad" Jul 22 02:21:17.609434 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.609404 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-165.ec2.internal" event="NodeReady" Jul 22 02:21:17.609621 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.609533 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 02:21:17.645383 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.645354 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp"] Jul 22 02:21:17.647572 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.647535 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.650744 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.650710 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 02:21:17.651727 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.651413 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.651727 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.651525 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7598849858-lz87x"] Jul 22 02:21:17.651727 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.651613 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 02:21:17.653030 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.653005 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-glzb5\"" Jul 22 02:21:17.653185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.653003 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.653582 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.653563 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-st5bj"] Jul 22 02:21:17.654640 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.654183 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.656346 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.655610 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm"] Jul 22 02:21:17.656346 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.655691 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.657353 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.657332 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 02:21:17.657455 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.657360 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c448899f8-d7r6m"] Jul 22 02:21:17.657808 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.657608 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.658307 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.658285 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 02:21:17.658388 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.658339 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rvkph\"" Jul 22 02:21:17.658388 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.658294 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 02:21:17.659107 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.659089 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pt7lk"] Jul 22 02:21:17.659223 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.659207 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.660208 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.660154 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.660393 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.660245 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.660653 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.660604 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.660847 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.660801 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t"] Jul 22 02:21:17.660985 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.660885 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.661113 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661084 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vn79z\"" Jul 22 02:21:17.661590 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661573 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 02:21:17.661783 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661667 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-58z5p\"" Jul 22 02:21:17.661917 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661760 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 02:21:17.661917 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661843 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 02:21:17.661917 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661855 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 02:21:17.662075 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.661925 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.662868 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.662729 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8"] Jul 22 02:21:17.662992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.662971 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:17.667905 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.667883 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.668488 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.668442 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2d779\"" Jul 22 02:21:17.670189 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.670004 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 02:21:17.672732 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.672677 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 02:21:17.672732 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.672701 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.673265 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.673044 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.673350 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.673293 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p"] Jul 22 02:21:17.673736 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.673717 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 02:21:17.673916 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.673891 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 02:21:17.674021 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.674003 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.675106 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.675077 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 02:21:17.675245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.674180 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.675297 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wwp6j\"" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.674287 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5m7qv\"" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.674237 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.674651 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.675523 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.675570 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.675585 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.676142 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.675910 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp"] Jul 22 02:21:17.677531 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.676687 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8"] Jul 22 02:21:17.678459 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.678440 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 02:21:17.680164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680147 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 02:21:17.680248 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680178 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.680323 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680251 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.680451 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680417 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mklgt\"" Jul 22 02:21:17.680451 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680440 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 02:21:17.680559 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680537 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qzqvx\"" Jul 22 02:21:17.680846 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680829 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 02:21:17.681913 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.680846 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.681913 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.681082 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.683668 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.683648 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 02:21:17.684211 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.683859 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c448899f8-d7r6m"] Jul 22 02:21:17.685267 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.685248 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-st5bj"] Jul 22 02:21:17.685441 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.685289 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7598849858-lz87x"] Jul 22 02:21:17.686917 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.686902 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-chnzz"] Jul 22 02:21:17.689538 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.689518 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gdf5m"] Jul 22 02:21:17.689975 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.689936 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.695245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.695226 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj"] Jul 22 02:21:17.695382 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.695368 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.696574 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.696550 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 02:21:17.696799 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.696550 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpzdk\"" Jul 22 02:21:17.696918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.696844 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 02:21:17.699218 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.698789 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t"] Jul 22 02:21:17.699312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.698893 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" Jul 22 02:21:17.700524 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.700505 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4hpjf\"" Jul 22 02:21:17.701452 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.700664 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.701452 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.700749 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 02:21:17.702500 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.702476 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.702840 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.702822 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dcm82\"" Jul 22 02:21:17.703228 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.703171 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:21:17.703334 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.703274 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:21:17.705664 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.705643 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj"] Jul 22 02:21:17.709338 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.709313 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gdf5m"] Jul 22 02:21:17.710538 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.710515 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-chnzz"] Jul 22 02:21:17.710955 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.710937 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm"] Jul 22 02:21:17.711482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.711462 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pt7lk"] Jul 22 02:21:17.712298 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.712278 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p"] Jul 22 02:21:17.811408 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-default-certificate\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.811563 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx6zk\" (UniqueName: \"kubernetes.io/projected/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-kube-api-access-cx6zk\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.811563 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5jt\" (UniqueName: \"kubernetes.io/projected/13da7e77-aeeb-4c25-a7ed-512fc1407a71-kube-api-access-6s5jt\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.811563 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/edc548b6-599d-469e-8362-aea863fa925d-config\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.811563 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-trusted-ca\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.811747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbq4h\" (UniqueName: \"kubernetes.io/projected/7a467496-665d-4f59-9b58-8d384fba5652-kube-api-access-dbq4h\") pod \"volume-data-source-validator-78cf4c6f6-tx6g8\" (UID: \"7a467496-665d-4f59-9b58-8d384fba5652\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" Jul 22 02:21:17.811747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811639 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ptl\" (UniqueName: \"kubernetes.io/projected/f7c31c9a-a2fc-4f93-a034-805ba7708a21-kube-api-access-l4ptl\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.811747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811666 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxpkj\" (UniqueName: \"kubernetes.io/projected/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-kube-api-access-fxpkj\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.811747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811716 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdlwf\" (UniqueName: \"kubernetes.io/projected/defc9a21-3c7c-49c8-9ced-49644ae3954b-kube-api-access-rdlwf\") pod \"ingress-canary-gdf5m\" (UID: \"defc9a21-3c7c-49c8-9ced-49644ae3954b\") " pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811783 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-bound-sa-token\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811813 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45gcn\" (UniqueName: \"kubernetes.io/projected/edc548b6-599d-469e-8362-aea863fa925d-kube-api-access-45gcn\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/defc9a21-3c7c-49c8-9ced-49644ae3954b-cert\") pod \"ingress-canary-gdf5m\" (UID: \"defc9a21-3c7c-49c8-9ced-49644ae3954b\") " pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgmld\" (UniqueName: \"kubernetes.io/projected/a5cd039e-20d2-45d5-a19c-8a9973b47ff0-kube-api-access-pgmld\") pod \"network-check-source-7bc5b6bf85-x7dbj\" (UID: \"a5cd039e-20d2-45d5-a19c-8a9973b47ff0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811919 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-config-volume\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.811962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811947 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811971 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/edc548b6-599d-469e-8362-aea863fa925d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.811994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-config\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812016 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-tmp-dir\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c31c9a-a2fc-4f93-a034-805ba7708a21-service-ca-bundle\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c23179af-03e0-4ab8-b9c8-e95baa2529d4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjwn8\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-kube-api-access-sjwn8\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812193 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edc31bf4-5231-4d96-8f32-08204dad564e-ca-trust-extracted\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.812225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812213 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f7c31c9a-a2fc-4f93-a034-805ba7708a21-snapshots\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812282 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-installation-pull-secrets\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812299 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-image-registry-private-configuration\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812319 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gvfz\" (UniqueName: \"kubernetes.io/projected/c23179af-03e0-4ab8-b9c8-e95baa2529d4-kube-api-access-5gvfz\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812344 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-trusted-ca\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812375 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-stats-auth\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812424 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-registry-certificates\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7c31c9a-a2fc-4f93-a034-805ba7708a21-tmp\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812504 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hws2f\" (UniqueName: \"kubernetes.io/projected/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-kube-api-access-hws2f\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812550 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.812587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.813032 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-serving-cert\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.813032 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7c31c9a-a2fc-4f93-a034-805ba7708a21-serving-cert\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.813032 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812688 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvpnb\" (UniqueName: \"kubernetes.io/projected/c34098ea-4889-4a79-8ee5-e4c78946bd7b-kube-api-access-nvpnb\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:17.813032 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812718 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.813032 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.812741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c31c9a-a2fc-4f93-a034-805ba7708a21-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.913520 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913433 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-bound-sa-token\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.913520 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45gcn\" (UniqueName: \"kubernetes.io/projected/edc548b6-599d-469e-8362-aea863fa925d-kube-api-access-45gcn\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.913520 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/defc9a21-3c7c-49c8-9ced-49644ae3954b-cert\") pod \"ingress-canary-gdf5m\" (UID: \"defc9a21-3c7c-49c8-9ced-49644ae3954b\") " pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913529 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmld\" (UniqueName: \"kubernetes.io/projected/a5cd039e-20d2-45d5-a19c-8a9973b47ff0-kube-api-access-pgmld\") pod \"network-check-source-7bc5b6bf85-x7dbj\" (UID: \"a5cd039e-20d2-45d5-a19c-8a9973b47ff0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913553 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-config-volume\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.913671 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.913752 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls podName:69f61c23-1cb1-4e11-8166-6ac5a80d71a5 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:18.413728252 +0000 UTC m=+37.049465061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls") pod "dns-default-chnzz" (UID: "69f61c23-1cb1-4e11-8166-6ac5a80d71a5") : secret "dns-default-metrics-tls" not found Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/edc548b6-599d-469e-8362-aea863fa925d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-config\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913892 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-tmp-dir\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913919 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c31c9a-a2fc-4f93-a034-805ba7708a21-service-ca-bundle\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913950 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.913977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c23179af-03e0-4ab8-b9c8-e95baa2529d4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914054 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjwn8\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-kube-api-access-sjwn8\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914080 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edc31bf4-5231-4d96-8f32-08204dad564e-ca-trust-extracted\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.914207 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914194 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f7c31c9a-a2fc-4f93-a034-805ba7708a21-snapshots\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914198 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-tmp-dir\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-installation-pull-secrets\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914258 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-config-volume\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914271 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-image-registry-private-configuration\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.914300 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gvfz\" (UniqueName: \"kubernetes.io/projected/c23179af-03e0-4ab8-b9c8-e95baa2529d4-kube-api-access-5gvfz\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-trusted-ca\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.914363 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls podName:c34098ea-4889-4a79-8ee5-e4c78946bd7b nodeName:}" failed. No retries permitted until 2026-07-22 02:21:18.414345084 +0000 UTC m=+37.050081886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9qw4t" (UID: "c34098ea-4889-4a79-8ee5-e4c78946bd7b") : secret "samples-operator-tls" not found Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914413 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-stats-auth\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914459 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-registry-certificates\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7c31c9a-a2fc-4f93-a034-805ba7708a21-tmp\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914514 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edc31bf4-5231-4d96-8f32-08204dad564e-ca-trust-extracted\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914518 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hws2f\" (UniqueName: \"kubernetes.io/projected/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-kube-api-access-hws2f\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.915065 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914594 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.914916 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c31c9a-a2fc-4f93-a034-805ba7708a21-service-ca-bundle\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.915049 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-config\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.915131 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.915450 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:18.415431946 +0000 UTC m=+37.051168745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : configmap references non-existent config key: service-ca.crt Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.915579 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c23179af-03e0-4ab8-b9c8-e95baa2529d4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.915665 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.916018 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.915987 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:17.916336 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.916041 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls podName:c23179af-03e0-4ab8-b9c8-e95baa2529d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:18.416026018 +0000 UTC m=+37.051762808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pldkp" (UID: "c23179af-03e0-4ab8-b9c8-e95baa2529d4") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:17.916336 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916329 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-registry-certificates\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.916557 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916532 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f7c31c9a-a2fc-4f93-a034-805ba7708a21-snapshots\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.916626 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916589 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7c31c9a-a2fc-4f93-a034-805ba7708a21-tmp\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.916836 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916813 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-serving-cert\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.916903 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916852 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7c31c9a-a2fc-4f93-a034-805ba7708a21-serving-cert\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.916903 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvpnb\" (UniqueName: \"kubernetes.io/projected/c34098ea-4889-4a79-8ee5-e4c78946bd7b-kube-api-access-nvpnb\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:17.916998 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.916998 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916941 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c31c9a-a2fc-4f93-a034-805ba7708a21-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.916998 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916971 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-default-certificate\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.917150 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.916997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx6zk\" (UniqueName: \"kubernetes.io/projected/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-kube-api-access-cx6zk\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.917150 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917040 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5jt\" (UniqueName: \"kubernetes.io/projected/13da7e77-aeeb-4c25-a7ed-512fc1407a71-kube-api-access-6s5jt\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.917150 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/edc548b6-599d-469e-8362-aea863fa925d-config\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.917150 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-trusted-ca\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.917150 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917145 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbq4h\" (UniqueName: \"kubernetes.io/projected/7a467496-665d-4f59-9b58-8d384fba5652-kube-api-access-dbq4h\") pod \"volume-data-source-validator-78cf4c6f6-tx6g8\" (UID: \"7a467496-665d-4f59-9b58-8d384fba5652\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" Jul 22 02:21:17.917391 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917174 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ptl\" (UniqueName: \"kubernetes.io/projected/f7c31c9a-a2fc-4f93-a034-805ba7708a21-kube-api-access-l4ptl\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.917391 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxpkj\" (UniqueName: \"kubernetes.io/projected/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-kube-api-access-fxpkj\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.917391 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917239 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdlwf\" (UniqueName: \"kubernetes.io/projected/defc9a21-3c7c-49c8-9ced-49644ae3954b-kube-api-access-rdlwf\") pod \"ingress-canary-gdf5m\" (UID: \"defc9a21-3c7c-49c8-9ced-49644ae3954b\") " pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.917391 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.917271 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-trusted-ca\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.917391 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.917322 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:18.417305567 +0000 UTC m=+37.053042356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : secret "router-metrics-certs-default" not found Jul 22 02:21:17.918132 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.918111 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:21:17.918132 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.918133 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7598849858-lz87x: secret "image-registry-tls" not found Jul 22 02:21:17.918276 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.918185 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.918276 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:17.918193 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls podName:edc31bf4-5231-4d96-8f32-08204dad564e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:18.418172104 +0000 UTC m=+37.053908904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls") pod "image-registry-7598849858-lz87x" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e") : secret "image-registry-tls" not found Jul 22 02:21:17.918392 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.918366 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/edc548b6-599d-469e-8362-aea863fa925d-config\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.922130 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.919246 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-image-registry-private-configuration\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.922130 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.919604 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-trusted-ca\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.922130 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.919933 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/defc9a21-3c7c-49c8-9ced-49644ae3954b-cert\") pod \"ingress-canary-gdf5m\" (UID: \"defc9a21-3c7c-49c8-9ced-49644ae3954b\") " pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.922130 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.920902 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c31c9a-a2fc-4f93-a034-805ba7708a21-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.922130 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.921429 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-stats-auth\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.922130 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.922087 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/edc548b6-599d-469e-8362-aea863fa925d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.922649 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.922624 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-serving-cert\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.924533 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.923736 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7c31c9a-a2fc-4f93-a034-805ba7708a21-serving-cert\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.927535 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.927510 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgmld\" (UniqueName: \"kubernetes.io/projected/a5cd039e-20d2-45d5-a19c-8a9973b47ff0-kube-api-access-pgmld\") pod \"network-check-source-7bc5b6bf85-x7dbj\" (UID: \"a5cd039e-20d2-45d5-a19c-8a9973b47ff0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" Jul 22 02:21:17.930172 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.930143 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gvfz\" (UniqueName: \"kubernetes.io/projected/c23179af-03e0-4ab8-b9c8-e95baa2529d4-kube-api-access-5gvfz\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:17.931880 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.931733 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ptl\" (UniqueName: \"kubernetes.io/projected/f7c31c9a-a2fc-4f93-a034-805ba7708a21-kube-api-access-l4ptl\") pod \"insights-operator-555d9d7885-st5bj\" (UID: \"f7c31c9a-a2fc-4f93-a034-805ba7708a21\") " pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.931967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.931753 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvpnb\" (UniqueName: \"kubernetes.io/projected/c34098ea-4889-4a79-8ee5-e4c78946bd7b-kube-api-access-nvpnb\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:17.931967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.931785 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5jt\" (UniqueName: \"kubernetes.io/projected/13da7e77-aeeb-4c25-a7ed-512fc1407a71-kube-api-access-6s5jt\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.931967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.931810 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx6zk\" (UniqueName: \"kubernetes.io/projected/4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3-kube-api-access-cx6zk\") pod \"kube-storage-version-migrator-operator-7658d9b749-2g47p\" (UID: \"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:17.932533 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.932489 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-installation-pull-secrets\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.933288 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.933224 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxpkj\" (UniqueName: \"kubernetes.io/projected/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-kube-api-access-fxpkj\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:17.933372 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.933300 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45gcn\" (UniqueName: \"kubernetes.io/projected/edc548b6-599d-469e-8362-aea863fa925d-kube-api-access-45gcn\") pod \"service-ca-operator-5bcbc94c78-mrjvm\" (UID: \"edc548b6-599d-469e-8362-aea863fa925d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:17.933372 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.933334 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-bound-sa-token\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.933482 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.933450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbq4h\" (UniqueName: \"kubernetes.io/projected/7a467496-665d-4f59-9b58-8d384fba5652-kube-api-access-dbq4h\") pod \"volume-data-source-validator-78cf4c6f6-tx6g8\" (UID: \"7a467496-665d-4f59-9b58-8d384fba5652\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" Jul 22 02:21:17.933538 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.933509 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjwn8\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-kube-api-access-sjwn8\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:17.933820 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.933708 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdlwf\" (UniqueName: \"kubernetes.io/projected/defc9a21-3c7c-49c8-9ced-49644ae3954b-kube-api-access-rdlwf\") pod \"ingress-canary-gdf5m\" (UID: \"defc9a21-3c7c-49c8-9ced-49644ae3954b\") " pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:17.934384 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.934361 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-default-certificate\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:17.935236 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.935215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hws2f\" (UniqueName: \"kubernetes.io/projected/17d6253b-91b2-43e4-b9fc-3744f8ce11ca-kube-api-access-hws2f\") pod \"console-operator-565b7b569d-pt7lk\" (UID: \"17d6253b-91b2-43e4-b9fc-3744f8ce11ca\") " pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:17.986924 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.986891 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-st5bj" Jul 22 02:21:17.995094 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:17.995073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" Jul 22 02:21:18.013253 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.013224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:18.027461 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.027411 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" Jul 22 02:21:18.034426 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.034409 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" Jul 22 02:21:18.048426 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.048405 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gdf5m" Jul 22 02:21:18.054529 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.054513 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" Jul 22 02:21:18.422398 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.422366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:18.422538 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.422407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:18.422538 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.422455 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:18.422538 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422479 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:19.422456792 +0000 UTC m=+38.058193598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : configmap references non-existent config key: service-ca.crt Jul 22 02:21:18.422538 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422533 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422543 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422575 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls podName:c34098ea-4889-4a79-8ee5-e4c78946bd7b nodeName:}" failed. No retries permitted until 2026-07-22 02:21:19.422562739 +0000 UTC m=+38.058299528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9qw4t" (UID: "c34098ea-4889-4a79-8ee5-e4c78946bd7b") : secret "samples-operator-tls" not found Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.422572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422598 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls podName:c23179af-03e0-4ab8-b9c8-e95baa2529d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:19.422581737 +0000 UTC m=+38.058318520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pldkp" (UID: "c23179af-03e0-4ab8-b9c8-e95baa2529d4") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422637 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422667 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:19.422659625 +0000 UTC m=+38.058396436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : secret "router-metrics-certs-default" not found Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.422661 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:18.422724 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422725 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:21:18.423081 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422736 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7598849858-lz87x: secret "image-registry-tls" not found Jul 22 02:21:18.423081 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.422740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:18.423081 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422788 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls podName:edc31bf4-5231-4d96-8f32-08204dad564e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:19.422759663 +0000 UTC m=+38.058496446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls") pod "image-registry-7598849858-lz87x" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e") : secret "image-registry-tls" not found Jul 22 02:21:18.423081 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422828 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:21:18.423081 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:18.422869 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls podName:69f61c23-1cb1-4e11-8166-6ac5a80d71a5 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:19.42285088 +0000 UTC m=+38.058587694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls") pod "dns-default-chnzz" (UID: "69f61c23-1cb1-4e11-8166-6ac5a80d71a5") : secret "dns-default-metrics-tls" not found Jul 22 02:21:18.865099 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.865062 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:18.865293 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.865062 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:18.865293 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.865062 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:18.868159 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.868121 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:21:18.869307 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.869284 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cw2sm\"" Jul 22 02:21:18.869444 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.869353 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:21:18.869525 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:18.869290 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8mll8\"" Jul 22 02:21:19.435810 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.434169 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:19.435810 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.434496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:19.435810 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.434687 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:21:19.435810 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.434704 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7598849858-lz87x: secret "image-registry-tls" not found Jul 22 02:21:19.435810 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.435677 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:21:19.436705 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.436465 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls podName:edc31bf4-5231-4d96-8f32-08204dad564e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:21.434749499 +0000 UTC m=+40.070486302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls") pod "image-registry-7598849858-lz87x" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e") : secret "image-registry-tls" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.436832 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls podName:69f61c23-1cb1-4e11-8166-6ac5a80d71a5 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:21.436808061 +0000 UTC m=+40.072544849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls") pod "dns-default-chnzz" (UID: "69f61c23-1cb1-4e11-8166-6ac5a80d71a5") : secret "dns-default-metrics-tls" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.436917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.436958 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.437002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.437067 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437092 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:21.437076676 +0000 UTC m=+40.072813482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : configmap references non-existent config key: service-ca.crt Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437140 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437175 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:21.437165313 +0000 UTC m=+40.072902100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : secret "router-metrics-certs-default" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437214 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437232 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437257 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls podName:c34098ea-4889-4a79-8ee5-e4c78946bd7b nodeName:}" failed. No retries permitted until 2026-07-22 02:21:21.437239036 +0000 UTC m=+40.072975838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9qw4t" (UID: "c34098ea-4889-4a79-8ee5-e4c78946bd7b") : secret "samples-operator-tls" not found Jul 22 02:21:19.437289 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:19.437273 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls podName:c23179af-03e0-4ab8-b9c8-e95baa2529d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:21.437263996 +0000 UTC m=+40.073000782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pldkp" (UID: "c23179af-03e0-4ab8-b9c8-e95baa2529d4") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:19.451930 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.451409 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pt7lk"] Jul 22 02:21:19.471012 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.470896 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-st5bj"] Jul 22 02:21:19.483314 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.483163 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p"] Jul 22 02:21:19.501027 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.500996 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm"] Jul 22 02:21:19.508145 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.508111 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gdf5m"] Jul 22 02:21:19.513865 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.513842 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8"] Jul 22 02:21:19.519070 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:19.519049 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj"] Jul 22 02:21:19.522568 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:19.522540 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6dd54b_a535_4fe1_b9e9_cd900c1fafa3.slice/crio-bb0d21c397dca30c9dbb331374d65d68346f2ef18a16c677fa98183d07062746 WatchSource:0}: Error finding container bb0d21c397dca30c9dbb331374d65d68346f2ef18a16c677fa98183d07062746: Status 404 returned error can't find the container with id bb0d21c397dca30c9dbb331374d65d68346f2ef18a16c677fa98183d07062746 Jul 22 02:21:19.562953 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:19.562688 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a467496_665d_4f59_9b58_8d384fba5652.slice/crio-490c23020a042886f23d1e5e31e15dd474edf2067c6f2ae980c5caa2cf12239e WatchSource:0}: Error finding container 490c23020a042886f23d1e5e31e15dd474edf2067c6f2ae980c5caa2cf12239e: Status 404 returned error can't find the container with id 490c23020a042886f23d1e5e31e15dd474edf2067c6f2ae980c5caa2cf12239e Jul 22 02:21:20.058936 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.058894 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gdf5m" event={"ID":"defc9a21-3c7c-49c8-9ced-49644ae3954b","Type":"ContainerStarted","Data":"34823a2881beeb04a68bc7ce15db40486cb61fa4b95702c6e3c36472d04e6d87"} Jul 22 02:21:20.063354 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.063322 2561 generic.go:356] "Generic (PLEG): container finished" podID="61382fc5-f560-4435-a9f0-d0dc961dbe30" containerID="debf98995857bd098f71a1b05f0bc43a3a92c516543bd22c90b7db201907c641" exitCode=0 Jul 22 02:21:20.063490 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.063402 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerDied","Data":"debf98995857bd098f71a1b05f0bc43a3a92c516543bd22c90b7db201907c641"} Jul 22 02:21:20.067758 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.066893 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" event={"ID":"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3","Type":"ContainerStarted","Data":"bb0d21c397dca30c9dbb331374d65d68346f2ef18a16c677fa98183d07062746"} Jul 22 02:21:20.070077 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.070046 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" event={"ID":"7a467496-665d-4f59-9b58-8d384fba5652","Type":"ContainerStarted","Data":"490c23020a042886f23d1e5e31e15dd474edf2067c6f2ae980c5caa2cf12239e"} Jul 22 02:21:20.072568 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.072115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-st5bj" event={"ID":"f7c31c9a-a2fc-4f93-a034-805ba7708a21","Type":"ContainerStarted","Data":"daac955f615b48d0f1d4cdc5e60fa74e2ecf57ce3b7cd56ee9335731c3484a2a"} Jul 22 02:21:20.078758 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.078699 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" event={"ID":"17d6253b-91b2-43e4-b9fc-3744f8ce11ca","Type":"ContainerStarted","Data":"2b0dfa6a3882b8e4c72b31bfb110eec432147de89a847a58d159f052708dcdde"} Jul 22 02:21:20.080191 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.080146 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" event={"ID":"a5cd039e-20d2-45d5-a19c-8a9973b47ff0","Type":"ContainerStarted","Data":"a0c905b5b8a944881d029cad325d5e3156dd4b1eb92bc7990fcc0a6d929feed7"} Jul 22 02:21:20.082816 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:20.082790 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" event={"ID":"edc548b6-599d-469e-8362-aea863fa925d","Type":"ContainerStarted","Data":"88b9ae2da850940896a137cb22a7712c20d7a0747cdf62c3013669aef89a814e"} Jul 22 02:21:21.094795 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.093975 2561 generic.go:356] "Generic (PLEG): container finished" podID="61382fc5-f560-4435-a9f0-d0dc961dbe30" containerID="7e2cdd39bfa4de2b3ef55660d65313fe7e318895c90322eafe9a345630ddfeba" exitCode=0 Jul 22 02:21:21.094795 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.094193 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerDied","Data":"7e2cdd39bfa4de2b3ef55660d65313fe7e318895c90322eafe9a345630ddfeba"} Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.457404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.457464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.457496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.457619 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.457661 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.457668 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls podName:c23179af-03e0-4ab8-b9c8-e95baa2529d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:25.45765492 +0000 UTC m=+44.093391702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pldkp" (UID: "c23179af-03e0-4ab8-b9c8-e95baa2529d4") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.457742 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:25.457723666 +0000 UTC m=+44.093460464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : secret "router-metrics-certs-default" not found Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.457945 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:21:21.457977 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.457963 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7598849858-lz87x: secret "image-registry-tls" not found Jul 22 02:21:21.458442 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.458004 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls podName:edc31bf4-5231-4d96-8f32-08204dad564e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:25.457989992 +0000 UTC m=+44.093726780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls") pod "image-registry-7598849858-lz87x" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e") : secret "image-registry-tls" not found Jul 22 02:21:21.458885 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.458638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:21.458885 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.458678 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:21.458885 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:21.458696 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:21.458885 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.458824 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:21:21.458885 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.458848 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:25.458834712 +0000 UTC m=+44.094571497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : configmap references non-existent config key: service-ca.crt Jul 22 02:21:21.458885 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.458880 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls podName:69f61c23-1cb1-4e11-8166-6ac5a80d71a5 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:25.458866315 +0000 UTC m=+44.094603101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls") pod "dns-default-chnzz" (UID: "69f61c23-1cb1-4e11-8166-6ac5a80d71a5") : secret "dns-default-metrics-tls" not found Jul 22 02:21:21.459237 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.458920 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:21:21.459237 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:21.458954 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls podName:c34098ea-4889-4a79-8ee5-e4c78946bd7b nodeName:}" failed. No retries permitted until 2026-07-22 02:21:25.458943433 +0000 UTC m=+44.094680220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9qw4t" (UID: "c34098ea-4889-4a79-8ee5-e4c78946bd7b") : secret "samples-operator-tls" not found Jul 22 02:21:22.108006 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.107087 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" event={"ID":"61382fc5-f560-4435-a9f0-d0dc961dbe30","Type":"ContainerStarted","Data":"aa77b1a440424dd1b211f155d5f5eba1aad900d4da8410cbdad01078a8c33b79"} Jul 22 02:21:22.111407 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.111375 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" event={"ID":"7a467496-665d-4f59-9b58-8d384fba5652","Type":"ContainerStarted","Data":"86e649a941719bc8b55ab750aed80df8d173c71ea42fba850366b601fc52b7a2"} Jul 22 02:21:22.135665 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.135606 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fgnn4" podStartSLOduration=8.16211697 podStartE2EDuration="40.135584586s" totalImagesPullingTime="31.973467616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:22.133725693 +0000 UTC m=+40.769462499" watchObservedRunningTime="2026-07-22 02:21:22.135584586 +0000 UTC m=+40.771321392" Jul 22 02:21:22.263922 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.263851 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tx6g8" podStartSLOduration=25.248452749 podStartE2EDuration="27.2638265s" totalImagesPullingTime="2.015373751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:22.14899304 +0000 UTC m=+40.784729846" watchObservedRunningTime="2026-07-22 02:21:22.2638265 +0000 UTC m=+40.899563304" Jul 22 02:21:22.264647 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.264624 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6"] Jul 22 02:21:22.282214 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.282172 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6"] Jul 22 02:21:22.282357 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.282286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:22.284941 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.284916 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 02:21:22.285251 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.285230 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 02:21:22.285473 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.285453 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-twtz7\"" Jul 22 02:21:22.474173 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.474135 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df32fe08-cfc7-4388-b46e-50d793f5522f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:22.474368 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.474198 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:22.575347 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.575309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df32fe08-cfc7-4388-b46e-50d793f5522f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:22.575551 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.575366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:22.575551 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:22.575484 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:21:22.575656 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:22.575565 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert podName:df32fe08-cfc7-4388-b46e-50d793f5522f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:23.075541374 +0000 UTC m=+41.711278172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-v5zg6" (UID: "df32fe08-cfc7-4388-b46e-50d793f5522f") : secret "networking-console-plugin-cert" not found Jul 22 02:21:22.576129 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:22.576099 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df32fe08-cfc7-4388-b46e-50d793f5522f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:23.079572 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:23.079469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:23.079879 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:23.079643 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:21:23.079879 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:23.079730 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert podName:df32fe08-cfc7-4388-b46e-50d793f5522f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:24.079706692 +0000 UTC m=+42.715443499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-v5zg6" (UID: "df32fe08-cfc7-4388-b46e-50d793f5522f") : secret "networking-console-plugin-cert" not found Jul 22 02:21:24.090160 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:24.090121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:24.090575 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:24.090272 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:21:24.090575 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:24.090350 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert podName:df32fe08-cfc7-4388-b46e-50d793f5522f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:26.090332781 +0000 UTC m=+44.726069563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-v5zg6" (UID: "df32fe08-cfc7-4388-b46e-50d793f5522f") : secret "networking-console-plugin-cert" not found Jul 22 02:21:25.502243 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:25.502205 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:25.502269 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:25.502302 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:25.502333 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502363 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:25.502386 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502433 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.50240963 +0000 UTC m=+52.138146432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : configmap references non-existent config key: service-ca.crt Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502437 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502463 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502475 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502499 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls podName:69f61c23-1cb1-4e11-8166-6ac5a80d71a5 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.502484179 +0000 UTC m=+52.138220962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls") pod "dns-default-chnzz" (UID: "69f61c23-1cb1-4e11-8166-6ac5a80d71a5") : secret "dns-default-metrics-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502522 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls podName:c34098ea-4889-4a79-8ee5-e4c78946bd7b nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.502512875 +0000 UTC m=+52.138249658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9qw4t" (UID: "c34098ea-4889-4a79-8ee5-e4c78946bd7b") : secret "samples-operator-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502557 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.502545733 +0000 UTC m=+52.138282522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : secret "router-metrics-certs-default" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502569 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls podName:c23179af-03e0-4ab8-b9c8-e95baa2529d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.502563368 +0000 UTC m=+52.138300151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pldkp" (UID: "c23179af-03e0-4ab8-b9c8-e95baa2529d4") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:25.502583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502642 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:21:25.502709 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502650 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7598849858-lz87x: secret "image-registry-tls" not found Jul 22 02:21:25.503388 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:25.502678 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls podName:edc31bf4-5231-4d96-8f32-08204dad564e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.502672219 +0000 UTC m=+52.138409002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls") pod "image-registry-7598849858-lz87x" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e") : secret "image-registry-tls" not found Jul 22 02:21:26.108581 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:26.108540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:26.108758 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:26.108724 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:21:26.108866 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:26.108831 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert podName:df32fe08-cfc7-4388-b46e-50d793f5522f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:30.108809947 +0000 UTC m=+48.744546749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-v5zg6" (UID: "df32fe08-cfc7-4388-b46e-50d793f5522f") : secret "networking-console-plugin-cert" not found Jul 22 02:21:28.130501 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.130458 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gdf5m" event={"ID":"defc9a21-3c7c-49c8-9ced-49644ae3954b","Type":"ContainerStarted","Data":"f52a84c135a12bc4c77de89fb3bf03811a9d993bd12ef377270970cd173368a2"} Jul 22 02:21:28.132181 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.132152 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" event={"ID":"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3","Type":"ContainerStarted","Data":"de684ae9d6d2cdd89a05c72bf552cbb80d7aaca72fbc5e5a20172303488bac89"} Jul 22 02:21:28.133653 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.133624 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-st5bj" event={"ID":"f7c31c9a-a2fc-4f93-a034-805ba7708a21","Type":"ContainerStarted","Data":"587f366934cf87c2a196470040ed15d64f34acc41d93b81552a5235f514834a1"} Jul 22 02:21:28.135760 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.135737 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/0.log" Jul 22 02:21:28.135885 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.135810 2561 generic.go:356] "Generic (PLEG): container finished" podID="17d6253b-91b2-43e4-b9fc-3744f8ce11ca" containerID="2098a6afc4ee3101129c361c79b3404771c87bdf334bff311a52d667e775ff15" exitCode=255 Jul 22 02:21:28.136185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.136043 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" event={"ID":"17d6253b-91b2-43e4-b9fc-3744f8ce11ca","Type":"ContainerDied","Data":"2098a6afc4ee3101129c361c79b3404771c87bdf334bff311a52d667e775ff15"} Jul 22 02:21:28.136185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.136130 2561 scope.go:117] "RemoveContainer" containerID="2098a6afc4ee3101129c361c79b3404771c87bdf334bff311a52d667e775ff15" Jul 22 02:21:28.138986 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.138669 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" event={"ID":"a5cd039e-20d2-45d5-a19c-8a9973b47ff0","Type":"ContainerStarted","Data":"e4bfd56128f0898572aa2ca82cc94565812ce0bc1fb987aad11eb4dc8870a4d4"} Jul 22 02:21:28.140590 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.140565 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" event={"ID":"edc548b6-599d-469e-8362-aea863fa925d","Type":"ContainerStarted","Data":"912c671886c81a14d752fd0fdda382c9427e6f3ed2ff72af5eee8dad923ee62d"} Jul 22 02:21:28.151755 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.151706 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gdf5m" podStartSLOduration=3.249040023 podStartE2EDuration="11.151691341s" totalImagesPullingTime="7.902651318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:28.150614799 +0000 UTC m=+46.786351606" watchObservedRunningTime="2026-07-22 02:21:28.151691341 +0000 UTC m=+46.787428143" Jul 22 02:21:28.169591 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.169531 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" podStartSLOduration=25.275678065 podStartE2EDuration="33.169510797s" totalImagesPullingTime="7.893832732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:28.168417589 +0000 UTC m=+46.804154397" watchObservedRunningTime="2026-07-22 02:21:28.169510797 +0000 UTC m=+46.805247601" Jul 22 02:21:28.205049 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.204996 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-x7dbj" podStartSLOduration=25.312668129 podStartE2EDuration="33.204974911s" totalImagesPullingTime="7.892306782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:28.203539729 +0000 UTC m=+46.839276536" watchObservedRunningTime="2026-07-22 02:21:28.204974911 +0000 UTC m=+46.840711717" Jul 22 02:21:28.222206 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.222150 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" podStartSLOduration=25.287946067 podStartE2EDuration="33.222128577s" totalImagesPullingTime="7.93418251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:28.220473787 +0000 UTC m=+46.856210596" watchObservedRunningTime="2026-07-22 02:21:28.222128577 +0000 UTC m=+46.857865385" Jul 22 02:21:28.248071 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:28.247204 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-st5bj" podStartSLOduration=25.354879755 podStartE2EDuration="33.247181665s" totalImagesPullingTime="7.89230191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:28.239371365 +0000 UTC m=+46.875108170" watchObservedRunningTime="2026-07-22 02:21:28.247181665 +0000 UTC m=+46.882918475" Jul 22 02:21:29.147503 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.147474 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:21:29.147994 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.147884 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/0.log" Jul 22 02:21:29.147994 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.147916 2561 generic.go:356] "Generic (PLEG): container finished" podID="17d6253b-91b2-43e4-b9fc-3744f8ce11ca" containerID="82c45d11a92282e1fb866839524e047f69a2cf64143191ffc2ae7e043f260ef7" exitCode=255 Jul 22 02:21:29.148108 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.147999 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" event={"ID":"17d6253b-91b2-43e4-b9fc-3744f8ce11ca","Type":"ContainerDied","Data":"82c45d11a92282e1fb866839524e047f69a2cf64143191ffc2ae7e043f260ef7"} Jul 22 02:21:29.148159 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.148116 2561 scope.go:117] "RemoveContainer" containerID="2098a6afc4ee3101129c361c79b3404771c87bdf334bff311a52d667e775ff15" Jul 22 02:21:29.148370 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.148348 2561 scope.go:117] "RemoveContainer" containerID="82c45d11a92282e1fb866839524e047f69a2cf64143191ffc2ae7e043f260ef7" Jul 22 02:21:29.148571 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:29.148549 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pt7lk_openshift-console-operator(17d6253b-91b2-43e4-b9fc-3744f8ce11ca)\"" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" podUID="17d6253b-91b2-43e4-b9fc-3744f8ce11ca" Jul 22 02:21:29.620962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.620926 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xp268"] Jul 22 02:21:29.653707 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.653681 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xp268"] Jul 22 02:21:29.653890 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.653830 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.656354 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.656328 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 02:21:29.656495 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.656382 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mzpsr\"" Jul 22 02:21:29.656495 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.656333 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 02:21:29.749377 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.749341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd608150-abbf-4063-8857-09d0b082205f-crio-socket\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.749561 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.749401 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrt8k\" (UniqueName: \"kubernetes.io/projected/bd608150-abbf-4063-8857-09d0b082205f-kube-api-access-mrt8k\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.749561 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.749492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.749561 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.749521 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.749732 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.749629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd608150-abbf-4063-8857-09d0b082205f-data-volume\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.850669 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.850633 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrt8k\" (UniqueName: \"kubernetes.io/projected/bd608150-abbf-4063-8857-09d0b082205f-kube-api-access-mrt8k\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.850896 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.850688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.850896 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.850707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.850896 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:29.850789 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:21:29.850896 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:29.850827 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:21:29.850896 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:29.850870 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:30.350849057 +0000 UTC m=+48.986585840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : configmap "kube-rbac-proxy" not found Jul 22 02:21:29.851159 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.850909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd608150-abbf-4063-8857-09d0b082205f-data-volume\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.851159 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:29.850938 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:30.350921828 +0000 UTC m=+48.986658612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : secret "insights-runtime-extractor-tls" not found Jul 22 02:21:29.851159 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.851010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd608150-abbf-4063-8857-09d0b082205f-crio-socket\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.851159 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.851131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd608150-abbf-4063-8857-09d0b082205f-crio-socket\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.851285 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.851165 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd608150-abbf-4063-8857-09d0b082205f-data-volume\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:29.862845 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:29.862819 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrt8k\" (UniqueName: \"kubernetes.io/projected/bd608150-abbf-4063-8857-09d0b082205f-kube-api-access-mrt8k\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:30.152295 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:30.152271 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:21:30.152779 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:30.152680 2561 scope.go:117] "RemoveContainer" containerID="82c45d11a92282e1fb866839524e047f69a2cf64143191ffc2ae7e043f260ef7" Jul 22 02:21:30.152915 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.152892 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pt7lk_openshift-console-operator(17d6253b-91b2-43e4-b9fc-3744f8ce11ca)\"" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" podUID="17d6253b-91b2-43e4-b9fc-3744f8ce11ca" Jul 22 02:21:30.153953 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:30.153930 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:30.154070 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.154055 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:21:30.154127 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.154117 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert podName:df32fe08-cfc7-4388-b46e-50d793f5522f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:38.154104267 +0000 UTC m=+56.789841050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-v5zg6" (UID: "df32fe08-cfc7-4388-b46e-50d793f5522f") : secret "networking-console-plugin-cert" not found Jul 22 02:21:30.356097 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:30.356060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:30.356097 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:30.356100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:30.356307 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.356183 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:21:30.356307 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.356233 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:21:30.356307 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.356247 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:31.35623157 +0000 UTC m=+49.991968353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : configmap "kube-rbac-proxy" not found Jul 22 02:21:30.356307 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:30.356272 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:31.356260955 +0000 UTC m=+49.991997738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : secret "insights-runtime-extractor-tls" not found Jul 22 02:21:31.366839 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:31.366791 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:31.366839 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:31.366846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:31.367348 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:31.366925 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:21:31.367348 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:31.367033 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:21:31.367348 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:31.367046 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.367023438 +0000 UTC m=+52.002760227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : configmap "kube-rbac-proxy" not found Jul 22 02:21:31.367348 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:31.367092 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:33.367079269 +0000 UTC m=+52.002816052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : secret "insights-runtime-extractor-tls" not found Jul 22 02:21:31.777129 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:31.777058 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-564zk_66dde4a9-a236-4bda-a7e2-2da7d71d668d/dns-node-resolver/0.log" Jul 22 02:21:33.178117 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.178089 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8z26_f29107fe-f689-4b7d-9b0b-39375612a435/node-ca/0.log" Jul 22 02:21:33.384104 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.384074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:33.384104 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.384106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:33.384338 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.384198 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:21:33.384338 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.384257 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:37.384240232 +0000 UTC m=+56.019977037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : configmap "kube-rbac-proxy" not found Jul 22 02:21:33.384450 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.384331 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:21:33.384450 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.384408 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:37.384389566 +0000 UTC m=+56.020126363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : secret "insights-runtime-extractor-tls" not found Jul 22 02:21:33.577374 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.577337 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdf5m_defc9a21-3c7c-49c8-9ced-49644ae3954b/serve-healthcheck-canary/0.log" Jul 22 02:21:33.585744 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.585719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:33.585875 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.585749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:33.585875 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.585846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:33.585967 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.585885 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:49.585867445 +0000 UTC m=+68.221604242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : configmap references non-existent config key: service-ca.crt Jul 22 02:21:33.585967 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.585933 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:33.585967 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.585934 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:21:33.586094 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.585971 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls podName:c23179af-03e0-4ab8-b9c8-e95baa2529d4 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:49.585959304 +0000 UTC m=+68.221696104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pldkp" (UID: "c23179af-03e0-4ab8-b9c8-e95baa2529d4") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:21:33.586094 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.585989 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls podName:c34098ea-4889-4a79-8ee5-e4c78946bd7b nodeName:}" failed. No retries permitted until 2026-07-22 02:21:49.585977412 +0000 UTC m=+68.221714195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9qw4t" (UID: "c34098ea-4889-4a79-8ee5-e4c78946bd7b") : secret "samples-operator-tls" not found Jul 22 02:21:33.586094 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.586055 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:33.586094 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.586088 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:33.586148 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586209 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586209 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586224 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7598849858-lz87x: secret "image-registry-tls" not found Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586259 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls podName:edc31bf4-5231-4d96-8f32-08204dad564e nodeName:}" failed. No retries permitted until 2026-07-22 02:21:49.586249161 +0000 UTC m=+68.221985944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls") pod "image-registry-7598849858-lz87x" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e") : secret "image-registry-tls" not found Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586217 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:21:33.586280 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586281 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs podName:13da7e77-aeeb-4c25-a7ed-512fc1407a71 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:49.586264981 +0000 UTC m=+68.222001779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs") pod "router-default-5c448899f8-d7r6m" (UID: "13da7e77-aeeb-4c25-a7ed-512fc1407a71") : secret "router-metrics-certs-default" not found Jul 22 02:21:33.586509 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:33.586302 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls podName:69f61c23-1cb1-4e11-8166-6ac5a80d71a5 nodeName:}" failed. No retries permitted until 2026-07-22 02:21:49.586292295 +0000 UTC m=+68.222029086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls") pod "dns-default-chnzz" (UID: "69f61c23-1cb1-4e11-8166-6ac5a80d71a5") : secret "dns-default-metrics-tls" not found Jul 22 02:21:34.379107 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:34.379069 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-2g47p_4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3/kube-storage-version-migrator-operator/0.log" Jul 22 02:21:37.419679 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:37.419640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:37.419679 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:37.419688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:37.420238 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:37.419899 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:21:37.420238 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:37.419977 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls podName:bd608150-abbf-4063-8857-09d0b082205f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:45.419960254 +0000 UTC m=+64.055697037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xp268" (UID: "bd608150-abbf-4063-8857-09d0b082205f") : secret "insights-runtime-extractor-tls" not found Jul 22 02:21:37.420238 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:37.420191 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd608150-abbf-4063-8857-09d0b082205f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:38.013951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:38.013917 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:38.014105 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:38.013974 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:38.014332 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:38.014319 2561 scope.go:117] "RemoveContainer" containerID="82c45d11a92282e1fb866839524e047f69a2cf64143191ffc2ae7e043f260ef7" Jul 22 02:21:38.014487 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:38.014467 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pt7lk_openshift-console-operator(17d6253b-91b2-43e4-b9fc-3744f8ce11ca)\"" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" podUID="17d6253b-91b2-43e4-b9fc-3744f8ce11ca" Jul 22 02:21:38.227046 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:38.227003 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:38.227214 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:38.227151 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 02:21:38.227294 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:21:38.227220 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert podName:df32fe08-cfc7-4388-b46e-50d793f5522f nodeName:}" failed. No retries permitted until 2026-07-22 02:21:54.227202667 +0000 UTC m=+72.862939451 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-v5zg6" (UID: "df32fe08-cfc7-4388-b46e-50d793f5522f") : secret "networking-console-plugin-cert" not found Jul 22 02:21:43.054663 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:43.054635 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gfnjk" Jul 22 02:21:45.492061 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:45.492017 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:45.494452 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:45.494422 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd608150-abbf-4063-8857-09d0b082205f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xp268\" (UID: \"bd608150-abbf-4063-8857-09d0b082205f\") " pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:45.565082 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:45.565050 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mzpsr\"" Jul 22 02:21:45.572964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:45.572937 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xp268" Jul 22 02:21:45.705628 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:45.705597 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xp268"] Jul 22 02:21:45.708485 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:45.708458 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd608150_abbf_4063_8857_09d0b082205f.slice/crio-057808cd6ccc2be24b32485ee8213ae6bb69bec7188b198f085236238578e4da WatchSource:0}: Error finding container 057808cd6ccc2be24b32485ee8213ae6bb69bec7188b198f085236238578e4da: Status 404 returned error can't find the container with id 057808cd6ccc2be24b32485ee8213ae6bb69bec7188b198f085236238578e4da Jul 22 02:21:46.205593 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:46.205480 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp268" event={"ID":"bd608150-abbf-4063-8857-09d0b082205f","Type":"ContainerStarted","Data":"057808cd6ccc2be24b32485ee8213ae6bb69bec7188b198f085236238578e4da"} Jul 22 02:21:47.513505 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.513467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:47.514017 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.513616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:47.516441 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.516409 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:21:47.516564 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.516451 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:21:47.527094 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.527071 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e149137e-7472-41b4-85a0-99c095e9db20-original-pull-secret\") pod \"global-pull-secret-syncer-k6hvt\" (UID: \"e149137e-7472-41b4-85a0-99c095e9db20\") " pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:47.527203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.527160 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3c2f13c-6b3c-4292-9b7d-8d184363dccb-metrics-certs\") pod \"network-metrics-daemon-n9wv2\" (UID: \"b3c2f13c-6b3c-4292-9b7d-8d184363dccb\") " pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:47.614617 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.614573 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:47.617655 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.617628 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7gzr\" (UniqueName: \"kubernetes.io/projected/3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad-kube-api-access-w7gzr\") pod \"network-check-target-rdd2m\" (UID: \"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad\") " pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:47.679201 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.679170 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cw2sm\"" Jul 22 02:21:47.685269 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.685242 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8mll8\"" Jul 22 02:21:47.686972 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.686946 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:47.688742 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.688719 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6hvt" Jul 22 02:21:47.693387 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.693367 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9wv2" Jul 22 02:21:47.844275 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:47.844190 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rdd2m"] Jul 22 02:21:48.070689 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:48.070653 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k6hvt"] Jul 22 02:21:48.073686 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:48.073662 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n9wv2"] Jul 22 02:21:49.631737 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.631706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:49.632272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.631758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:49.632272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.631843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:49.632272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.631888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:49.632272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.632151 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:49.632272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.632211 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:49.632929 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.632868 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13da7e77-aeeb-4c25-a7ed-512fc1407a71-service-ca-bundle\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:49.634870 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.634821 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da7e77-aeeb-4c25-a7ed-512fc1407a71-metrics-certs\") pod \"router-default-5c448899f8-d7r6m\" (UID: \"13da7e77-aeeb-4c25-a7ed-512fc1407a71\") " pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:49.635467 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.635440 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"image-registry-7598849858-lz87x\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:49.635546 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.635454 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c23179af-03e0-4ab8-b9c8-e95baa2529d4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pldkp\" (UID: \"c23179af-03e0-4ab8-b9c8-e95baa2529d4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:49.635546 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.635454 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c34098ea-4889-4a79-8ee5-e4c78946bd7b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9qw4t\" (UID: \"c34098ea-4889-4a79-8ee5-e4c78946bd7b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:49.636332 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.636313 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69f61c23-1cb1-4e11-8166-6ac5a80d71a5-metrics-tls\") pod \"dns-default-chnzz\" (UID: \"69f61c23-1cb1-4e11-8166-6ac5a80d71a5\") " pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:49.764101 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.764066 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-glzb5\"" Jul 22 02:21:49.772167 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.772150 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" Jul 22 02:21:49.783800 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.780010 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rvkph\"" Jul 22 02:21:49.788011 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.787986 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:49.806603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.806584 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2d779\"" Jul 22 02:21:49.814592 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.814569 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:49.822815 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.822798 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wwp6j\"" Jul 22 02:21:49.831828 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.831810 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" Jul 22 02:21:49.844349 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.844330 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpzdk\"" Jul 22 02:21:49.853114 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:49.853094 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-chnzz" Jul 22 02:21:50.225809 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:50.225750 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c2f13c_6b3c_4292_9b7d_8d184363dccb.slice/crio-0a90caef6aa2ec48a87231b13301c6ddc5f78b12acc0dcd1d3f93ae243667a29 WatchSource:0}: Error finding container 0a90caef6aa2ec48a87231b13301c6ddc5f78b12acc0dcd1d3f93ae243667a29: Status 404 returned error can't find the container with id 0a90caef6aa2ec48a87231b13301c6ddc5f78b12acc0dcd1d3f93ae243667a29 Jul 22 02:21:50.227388 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:50.227358 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode149137e_7472_41b4_85a0_99c095e9db20.slice/crio-efb0b87292eae8dc0f95b9e0b63de381174e64ef23d9636ab93d119ea1606bac WatchSource:0}: Error finding container efb0b87292eae8dc0f95b9e0b63de381174e64ef23d9636ab93d119ea1606bac: Status 404 returned error can't find the container with id efb0b87292eae8dc0f95b9e0b63de381174e64ef23d9636ab93d119ea1606bac Jul 22 02:21:50.227675 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:50.227589 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2ffec3_918f_41d9_a9ea_c244f5c3f9ad.slice/crio-c318ffa990455427ffcea48cd4858d76522e56528c31d8420e059758a769466c WatchSource:0}: Error finding container c318ffa990455427ffcea48cd4858d76522e56528c31d8420e059758a769466c: Status 404 returned error can't find the container with id c318ffa990455427ffcea48cd4858d76522e56528c31d8420e059758a769466c Jul 22 02:21:51.233057 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.233018 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k6hvt" event={"ID":"e149137e-7472-41b4-85a0-99c095e9db20","Type":"ContainerStarted","Data":"efb0b87292eae8dc0f95b9e0b63de381174e64ef23d9636ab93d119ea1606bac"} Jul 22 02:21:51.241854 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.241818 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rdd2m" event={"ID":"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad","Type":"ContainerStarted","Data":"c318ffa990455427ffcea48cd4858d76522e56528c31d8420e059758a769466c"} Jul 22 02:21:51.259338 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.258460 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9wv2" event={"ID":"b3c2f13c-6b3c-4292-9b7d-8d184363dccb","Type":"ContainerStarted","Data":"0a90caef6aa2ec48a87231b13301c6ddc5f78b12acc0dcd1d3f93ae243667a29"} Jul 22 02:21:51.373393 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.373280 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp"] Jul 22 02:21:51.398212 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.398159 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c448899f8-d7r6m"] Jul 22 02:21:51.398532 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.398505 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t"] Jul 22 02:21:51.583709 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:51.583660 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23179af_03e0_4ab8_b9c8_e95baa2529d4.slice/crio-95dd5e9a866cb73cc7b7194e35d241e14149aee6a0d83e463dd0e12245fbbc5c WatchSource:0}: Error finding container 95dd5e9a866cb73cc7b7194e35d241e14149aee6a0d83e463dd0e12245fbbc5c: Status 404 returned error can't find the container with id 95dd5e9a866cb73cc7b7194e35d241e14149aee6a0d83e463dd0e12245fbbc5c Jul 22 02:21:51.585633 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:51.585589 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13da7e77_aeeb_4c25_a7ed_512fc1407a71.slice/crio-4ea602640dc1fc469bfd622b4c53c723645cb9c97d921647546a9ae0b098c541 WatchSource:0}: Error finding container 4ea602640dc1fc469bfd622b4c53c723645cb9c97d921647546a9ae0b098c541: Status 404 returned error can't find the container with id 4ea602640dc1fc469bfd622b4c53c723645cb9c97d921647546a9ae0b098c541 Jul 22 02:21:51.645850 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.645699 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-chnzz"] Jul 22 02:21:51.646726 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.646689 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7598849858-lz87x"] Jul 22 02:21:51.655734 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:51.654311 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc31bf4_5231_4d96_8f32_08204dad564e.slice/crio-67b377417d563cc865ba4efca64ec990bc129b144deaa415337b6f39138a95d0 WatchSource:0}: Error finding container 67b377417d563cc865ba4efca64ec990bc129b144deaa415337b6f39138a95d0: Status 404 returned error can't find the container with id 67b377417d563cc865ba4efca64ec990bc129b144deaa415337b6f39138a95d0 Jul 22 02:21:51.868395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:51.868251 2561 scope.go:117] "RemoveContainer" containerID="82c45d11a92282e1fb866839524e047f69a2cf64143191ffc2ae7e043f260ef7" Jul 22 02:21:52.275447 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.274536 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rdd2m" event={"ID":"3a2ffec3-918f-41d9-a9ea-c244f5c3f9ad","Type":"ContainerStarted","Data":"02e33275b725d9d500d94f28054c81d3d113a11ba99d15ef6adbecd261981952"} Jul 22 02:21:52.275447 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.275350 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:21:52.283103 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.282456 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7598849858-lz87x" event={"ID":"edc31bf4-5231-4d96-8f32-08204dad564e","Type":"ContainerStarted","Data":"193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64"} Jul 22 02:21:52.283103 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.282505 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7598849858-lz87x" event={"ID":"edc31bf4-5231-4d96-8f32-08204dad564e","Type":"ContainerStarted","Data":"67b377417d563cc865ba4efca64ec990bc129b144deaa415337b6f39138a95d0"} Jul 22 02:21:52.283103 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.283056 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:21:52.291411 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.291089 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rdd2m" podStartSLOduration=69.291067588 podStartE2EDuration="1m9.291067588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:52.290923038 +0000 UTC m=+70.926659845" watchObservedRunningTime="2026-07-22 02:21:52.291067588 +0000 UTC m=+70.926804385" Jul 22 02:21:52.305662 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.301806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" event={"ID":"c23179af-03e0-4ab8-b9c8-e95baa2529d4","Type":"ContainerStarted","Data":"95dd5e9a866cb73cc7b7194e35d241e14149aee6a0d83e463dd0e12245fbbc5c"} Jul 22 02:21:52.321374 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.321285 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7598849858-lz87x" podStartSLOduration=69.321264307 podStartE2EDuration="1m9.321264307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:52.320486208 +0000 UTC m=+70.956223018" watchObservedRunningTime="2026-07-22 02:21:52.321264307 +0000 UTC m=+70.957001107" Jul 22 02:21:52.324326 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.322656 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:21:52.324326 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.322743 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" event={"ID":"17d6253b-91b2-43e4-b9fc-3744f8ce11ca","Type":"ContainerStarted","Data":"25eb750ed8b9ec14bcdf26f60aad3b74e6262cdfead0d61ae3f07d2418b56529"} Jul 22 02:21:52.324326 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.323519 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:52.329265 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.329201 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp268" event={"ID":"bd608150-abbf-4063-8857-09d0b082205f","Type":"ContainerStarted","Data":"97d9b55b332001e40aba8f022831c36ee35e15703d411c07a884de9371eb10ae"} Jul 22 02:21:52.333117 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.332510 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9wv2" event={"ID":"b3c2f13c-6b3c-4292-9b7d-8d184363dccb","Type":"ContainerStarted","Data":"04786bf09fd4ea0c3d706ac57af0605e593f9cfb820d4f847e8bdc579d7033df"} Jul 22 02:21:52.333117 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.332570 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9wv2" event={"ID":"b3c2f13c-6b3c-4292-9b7d-8d184363dccb","Type":"ContainerStarted","Data":"90dfe7a8100eda1d7447c51bce6d698a271eafadd7e5890c728a213e167022b6"} Jul 22 02:21:52.334698 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.334637 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-chnzz" event={"ID":"69f61c23-1cb1-4e11-8166-6ac5a80d71a5","Type":"ContainerStarted","Data":"ebe302ecad59c3b7216b3ba26f2955baa3b453cb6ded1ffc934cae083b743343"} Jul 22 02:21:52.339146 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.338926 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" event={"ID":"c34098ea-4889-4a79-8ee5-e4c78946bd7b","Type":"ContainerStarted","Data":"e6247df4638cdea777a9fd3014c8fcde08ad2200160cd1c0be0c3ec5496a0f74"} Jul 22 02:21:52.341305 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.341255 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" podStartSLOduration=49.447813188 podStartE2EDuration="57.341238082s" totalImagesPullingTime="7.893424894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:52.340064575 +0000 UTC m=+70.975801380" watchObservedRunningTime="2026-07-22 02:21:52.341238082 +0000 UTC m=+70.976974889" Jul 22 02:21:52.348403 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.348374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c448899f8-d7r6m" event={"ID":"13da7e77-aeeb-4c25-a7ed-512fc1407a71","Type":"ContainerStarted","Data":"a3223d6c9269c2712e36d592d0f6eb7db4b99ea99d8a307697d53ab23471fbfe"} Jul 22 02:21:52.348607 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.348425 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c448899f8-d7r6m" event={"ID":"13da7e77-aeeb-4c25-a7ed-512fc1407a71","Type":"ContainerStarted","Data":"4ea602640dc1fc469bfd622b4c53c723645cb9c97d921647546a9ae0b098c541"} Jul 22 02:21:52.356916 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.356861 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n9wv2" podStartSLOduration=68.968023039 podStartE2EDuration="1m10.356847199s" totalImagesPullingTime="1.38882416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:52.355135772 +0000 UTC m=+70.990872578" watchObservedRunningTime="2026-07-22 02:21:52.356847199 +0000 UTC m=+70.992584007" Jul 22 02:21:52.372957 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.372905 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5c448899f8-d7r6m" podStartSLOduration=57.372886531 podStartE2EDuration="57.372886531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:52.371718182 +0000 UTC m=+71.007454989" watchObservedRunningTime="2026-07-22 02:21:52.372886531 +0000 UTC m=+71.008623340" Jul 22 02:21:52.815485 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.815407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:52.818573 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:52.818349 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:53.324525 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:53.324481 2561 patch_prober.go:28] interesting pod/console-operator-565b7b569d-pt7lk container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/readyz\": context deadline exceeded" start-of-body= Jul 22 02:21:53.325068 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:53.324561 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" podUID="17d6253b-91b2-43e4-b9fc-3744f8ce11ca" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.16:8443/readyz\": context deadline exceeded" Jul 22 02:21:53.363200 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:53.361602 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:53.364148 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:53.363933 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c448899f8-d7r6m" Jul 22 02:21:53.474559 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:53.474324 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-pt7lk" Jul 22 02:21:54.286132 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:54.286087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:54.289040 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:54.289014 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df32fe08-cfc7-4388-b46e-50d793f5522f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-v5zg6\" (UID: \"df32fe08-cfc7-4388-b46e-50d793f5522f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:54.398071 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:54.398044 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-twtz7\"" Jul 22 02:21:54.406149 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:54.406123 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" Jul 22 02:21:59.258783 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.258739 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6"] Jul 22 02:21:59.261135 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:21:59.261110 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf32fe08_cfc7_4388_b46e_50d793f5522f.slice/crio-16269f98e0e76c80d3410562e088bccb3ec0aae9ede19907f519a037e8d54cdd WatchSource:0}: Error finding container 16269f98e0e76c80d3410562e088bccb3ec0aae9ede19907f519a037e8d54cdd: Status 404 returned error can't find the container with id 16269f98e0e76c80d3410562e088bccb3ec0aae9ede19907f519a037e8d54cdd Jul 22 02:21:59.387476 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.387389 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-chnzz" event={"ID":"69f61c23-1cb1-4e11-8166-6ac5a80d71a5","Type":"ContainerStarted","Data":"2317cc6bcdb2f624e365861f469bc94fde77214d33d04679994b317ef739f4c6"} Jul 22 02:21:59.389216 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.389160 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" event={"ID":"c34098ea-4889-4a79-8ee5-e4c78946bd7b","Type":"ContainerStarted","Data":"eae77b857eeffdf1e26c4c52374105bfe1b351bf6298f6e2c0aa6ffa55c3d123"} Jul 22 02:21:59.390806 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.390441 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k6hvt" event={"ID":"e149137e-7472-41b4-85a0-99c095e9db20","Type":"ContainerStarted","Data":"3a2e001f8c787534f320bfc7b5326b2c831c40aee051c6732c2bc4220706d02b"} Jul 22 02:21:59.392996 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.392150 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" event={"ID":"c23179af-03e0-4ab8-b9c8-e95baa2529d4","Type":"ContainerStarted","Data":"eba006bcc499ed5c1ee063a9bfc27a7d868320ffa4503aea8cd940410aba9e0e"} Jul 22 02:21:59.396809 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.396072 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" event={"ID":"df32fe08-cfc7-4388-b46e-50d793f5522f","Type":"ContainerStarted","Data":"16269f98e0e76c80d3410562e088bccb3ec0aae9ede19907f519a037e8d54cdd"} Jul 22 02:21:59.407247 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.407185 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k6hvt" podStartSLOduration=67.508409978 podStartE2EDuration="1m16.407168289s" totalImagesPullingTime="8.898758311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:59.405908224 +0000 UTC m=+78.041645031" watchObservedRunningTime="2026-07-22 02:21:59.407168289 +0000 UTC m=+78.042905095" Jul 22 02:21:59.424784 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:21:59.424705 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pldkp" podStartSLOduration=56.896500729 podStartE2EDuration="1m4.424689361s" totalImagesPullingTime="7.528188632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:59.423203355 +0000 UTC m=+78.058940159" watchObservedRunningTime="2026-07-22 02:21:59.424689361 +0000 UTC m=+78.060426168" Jul 22 02:22:00.404536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:00.404414 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp268" event={"ID":"bd608150-abbf-4063-8857-09d0b082205f","Type":"ContainerStarted","Data":"1723afaab2f75dfa37a8117c583524a89f2ad0642b9ecf2936c866c5cfe44efc"} Jul 22 02:22:00.408083 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:00.407998 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-chnzz" event={"ID":"69f61c23-1cb1-4e11-8166-6ac5a80d71a5","Type":"ContainerStarted","Data":"940e8a111511ecce2aedd6dde35b24f58b6938c57ca801b6bbdf44dc740c3613"} Jul 22 02:22:00.408237 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:00.408186 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-chnzz" Jul 22 02:22:00.411812 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:00.411788 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" event={"ID":"c34098ea-4889-4a79-8ee5-e4c78946bd7b","Type":"ContainerStarted","Data":"96c3598f63d2014dcf83e5b07c29c595608d1a56cc1f3d7fd258466dc228078d"} Jul 22 02:22:00.443162 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:00.443114 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-chnzz" podStartSLOduration=35.985098996 podStartE2EDuration="43.443093625s" totalImagesPullingTime="7.457994629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:00.42686163 +0000 UTC m=+79.062598442" watchObservedRunningTime="2026-07-22 02:22:00.443093625 +0000 UTC m=+79.078830429" Jul 22 02:22:01.416751 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:01.416709 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" event={"ID":"df32fe08-cfc7-4388-b46e-50d793f5522f","Type":"ContainerStarted","Data":"d7fe3aeb6a9425df63dd5f2e070b730f8abb915a8e8c90c2327fee7e80d57a75"} Jul 22 02:22:01.432882 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:01.432834 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-v5zg6" podStartSLOduration=37.682353937 podStartE2EDuration="39.432813959s" totalImagesPullingTime="1.750460022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:01.432655964 +0000 UTC m=+80.068392770" watchObservedRunningTime="2026-07-22 02:22:01.432813959 +0000 UTC m=+80.068550764" Jul 22 02:22:01.433627 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:01.433595 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9qw4t" podStartSLOduration=58.956379546 podStartE2EDuration="1m6.433584118s" totalImagesPullingTime="7.477204572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:00.442906546 +0000 UTC m=+79.078643352" watchObservedRunningTime="2026-07-22 02:22:01.433584118 +0000 UTC m=+80.069320923" Jul 22 02:22:04.434954 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:04.434900 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xp268" event={"ID":"bd608150-abbf-4063-8857-09d0b082205f","Type":"ContainerStarted","Data":"828621e59bbb068744c79c18a88b155cedc75bf7539c8b17928ec00a95114941"} Jul 22 02:22:04.457005 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:04.456960 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xp268" podStartSLOduration=17.959927999 podStartE2EDuration="35.456944965s" totalImagesPullingTime="17.497016966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:21:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:04.454981298 +0000 UTC m=+83.090718103" watchObservedRunningTime="2026-07-22 02:22:04.456944965 +0000 UTC m=+83.092681770" Jul 22 02:22:10.419388 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:10.419327 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-chnzz" Jul 22 02:22:14.369672 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:14.369643 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:22:16.192527 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:16.192495 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7598849858-lz87x"] Jul 22 02:22:20.158871 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.158841 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dzx8b"] Jul 22 02:22:20.177365 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.177338 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.179730 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.179710 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 02:22:20.179891 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.179709 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 02:22:20.179891 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.179753 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 02:22:20.180112 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.180093 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 02:22:20.180710 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.180684 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjx6g\"" Jul 22 02:22:20.298807 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.298774 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-textfile\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.298951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.298835 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9d1298d-8de9-483e-9ca1-52d3a12d8711-metrics-client-ca\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.298951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.298867 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjcxj\" (UniqueName: \"kubernetes.io/projected/d9d1298d-8de9-483e-9ca1-52d3a12d8711-kube-api-access-hjcxj\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.298951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.298894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-root\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.298951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.298936 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-tls\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.298951 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.298950 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-wtmp\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.299115 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.299008 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.299115 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.299038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-sys\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.299115 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.299057 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399516 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399485 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-sys\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399516 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399699 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-textfile\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399699 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399566 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9d1298d-8de9-483e-9ca1-52d3a12d8711-metrics-client-ca\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399699 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399594 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-sys\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399699 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjcxj\" (UniqueName: \"kubernetes.io/projected/d9d1298d-8de9-483e-9ca1-52d3a12d8711-kube-api-access-hjcxj\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399699 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399670 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-root\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399732 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-tls\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399750 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-wtmp\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399800 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-root\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399919 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-textfile\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.399964 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.399942 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-wtmp\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.400344 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.400296 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9d1298d-8de9-483e-9ca1-52d3a12d8711-metrics-client-ca\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.400344 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.400341 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-accelerators-collector-config\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.402042 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.402015 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.402201 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.402182 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9d1298d-8de9-483e-9ca1-52d3a12d8711-node-exporter-tls\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.411987 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.411940 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjcxj\" (UniqueName: \"kubernetes.io/projected/d9d1298d-8de9-483e-9ca1-52d3a12d8711-kube-api-access-hjcxj\") pod \"node-exporter-dzx8b\" (UID: \"d9d1298d-8de9-483e-9ca1-52d3a12d8711\") " pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.486505 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:20.486475 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dzx8b" Jul 22 02:22:20.498530 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:22:20.498497 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d1298d_8de9_483e_9ca1_52d3a12d8711.slice/crio-731c8d33f58aa2b49d87b06fd88b1ad40a8daf3357438375128e303a206d1d59 WatchSource:0}: Error finding container 731c8d33f58aa2b49d87b06fd88b1ad40a8daf3357438375128e303a206d1d59: Status 404 returned error can't find the container with id 731c8d33f58aa2b49d87b06fd88b1ad40a8daf3357438375128e303a206d1d59 Jul 22 02:22:21.496302 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:21.496273 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzx8b" event={"ID":"d9d1298d-8de9-483e-9ca1-52d3a12d8711","Type":"ContainerStarted","Data":"37143ad03f1c4fe2b4cc9b41cb60a748e4faa99d96e662cacbdba3eb48e90d71"} Jul 22 02:22:21.496576 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:21.496362 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzx8b" event={"ID":"d9d1298d-8de9-483e-9ca1-52d3a12d8711","Type":"ContainerStarted","Data":"731c8d33f58aa2b49d87b06fd88b1ad40a8daf3357438375128e303a206d1d59"} Jul 22 02:22:22.500963 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:22.500928 2561 generic.go:356] "Generic (PLEG): container finished" podID="d9d1298d-8de9-483e-9ca1-52d3a12d8711" containerID="37143ad03f1c4fe2b4cc9b41cb60a748e4faa99d96e662cacbdba3eb48e90d71" exitCode=0 Jul 22 02:22:22.501355 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:22.501003 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzx8b" event={"ID":"d9d1298d-8de9-483e-9ca1-52d3a12d8711","Type":"ContainerDied","Data":"37143ad03f1c4fe2b4cc9b41cb60a748e4faa99d96e662cacbdba3eb48e90d71"} Jul 22 02:22:23.507064 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:23.507031 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzx8b" event={"ID":"d9d1298d-8de9-483e-9ca1-52d3a12d8711","Type":"ContainerStarted","Data":"d36beff5b238e9f501d8ce31d9d8607f118cf33240d7c92bf95286fd007875f6"} Jul 22 02:22:23.507552 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:23.507098 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dzx8b" event={"ID":"d9d1298d-8de9-483e-9ca1-52d3a12d8711","Type":"ContainerStarted","Data":"b94158ca4326da65a7b4237c532f847f6875e211739579dea29f233b44f4b734"} Jul 22 02:22:23.528405 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:23.528360 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dzx8b" podStartSLOduration=2.549117099 podStartE2EDuration="3.52834537s" totalImagesPullingTime="905.558747ms" totalInitContainerRuntime="73.669524ms" isStatefulPod=true podCreationTimestamp="2026-07-22 02:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:23.527102361 +0000 UTC m=+102.162839166" watchObservedRunningTime="2026-07-22 02:22:23.52834537 +0000 UTC m=+102.164082174" Jul 22 02:22:24.367960 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:24.367929 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rdd2m" Jul 22 02:22:26.423703 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.423666 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 02:22:26.465135 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.465107 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 02:22:26.465280 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.465233 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.467920 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.467896 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 22 02:22:26.468054 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.467900 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 22 02:22:26.468124 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468074 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kt6q9\"" Jul 22 02:22:26.468238 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468224 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 22 02:22:26.468328 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468290 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 22 02:22:26.468390 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468321 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 22 02:22:26.468390 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468376 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 22 02:22:26.468861 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468843 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 22 02:22:26.468861 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468858 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 02:22:26.469020 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468948 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 22 02:22:26.469020 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468955 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c2o6vu84tqln1\"" Jul 22 02:22:26.469020 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.468965 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 22 02:22:26.469231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.469077 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 22 02:22:26.469231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.469088 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 22 02:22:26.469302 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.469281 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 22 02:22:26.471089 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.471072 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 22 02:22:26.474273 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.474256 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 22 02:22:26.552658 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.552658 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552658 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.552871 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.552871 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552695 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.552871 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552722 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.552871 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552803 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.552871 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbkc2\" (UniqueName: \"kubernetes.io/projected/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-kube-api-access-bbkc2\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553023 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552879 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553023 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553023 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552946 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553023 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.552982 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553023 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553023 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553019 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-config\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.553203 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.553192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653664 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653664 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653664 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653689 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-config\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653738 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653807 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.653918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653881 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654147 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653937 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654147 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653969 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654147 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.653997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654643 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654700 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654790 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbkc2\" (UniqueName: \"kubernetes.io/projected/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-kube-api-access-bbkc2\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654896 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.654967 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655470 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.654991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655470 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.655031 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655470 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.655057 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655470 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.655080 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655470 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.655106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655470 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.655415 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.655686 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.655595 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.657091 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.657030 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.657428 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.657402 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.659323 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659296 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.659427 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659387 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.659600 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659576 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.659668 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659602 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.659726 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659666 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-config\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.659726 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659684 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.660024 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.659992 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.660210 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.660188 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.660258 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.660216 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.660329 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.660313 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.660627 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.660607 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.661079 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.661054 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.663720 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.663696 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbkc2\" (UniqueName: \"kubernetes.io/projected/a8a64a0d-ebba-4dbd-a625-6e62579b1bb2-kube-api-access-bbkc2\") pod \"prometheus-k8s-0\" (UID: \"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.775986 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.775903 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:26.945678 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:26.945634 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 02:22:26.948523 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:22:26.948480 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a64a0d_ebba_4dbd_a625_6e62579b1bb2.slice/crio-0a2626957464dfb4f3bf2cdc2491ac39e9bd9f396b861fc80b068dc68843e63e WatchSource:0}: Error finding container 0a2626957464dfb4f3bf2cdc2491ac39e9bd9f396b861fc80b068dc68843e63e: Status 404 returned error can't find the container with id 0a2626957464dfb4f3bf2cdc2491ac39e9bd9f396b861fc80b068dc68843e63e Jul 22 02:22:27.525691 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:27.525651 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"0a2626957464dfb4f3bf2cdc2491ac39e9bd9f396b861fc80b068dc68843e63e"} Jul 22 02:22:28.530791 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:28.530684 2561 generic.go:356] "Generic (PLEG): container finished" podID="a8a64a0d-ebba-4dbd-a625-6e62579b1bb2" containerID="da524d5289ea109c324132cdc4e4028afb2a549f094d6d451120e63f4ba8d22b" exitCode=0 Jul 22 02:22:28.531173 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:28.530788 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerDied","Data":"da524d5289ea109c324132cdc4e4028afb2a549f094d6d451120e63f4ba8d22b"} Jul 22 02:22:32.549044 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:32.549008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"53a1e543d6c24127d5bac138974bbc5e10d1391db33390a74e7ec1b3f192df53"} Jul 22 02:22:32.549409 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:32.549072 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"3339cec10e99f6bfe6bb0d0bcc006d3aac6d4e3a3ba30c0e050cb45782556c64"} Jul 22 02:22:34.560588 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:34.560558 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"fc00b3420c955683fd6cb514b880e30486338bac16a84e1ec69ffe83179520d8"} Jul 22 02:22:34.560985 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:34.560635 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"c91ea45e01db881f0e65b2c371137aadec7023ec3d2dfb85a9a4f6685120294d"} Jul 22 02:22:34.562278 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:34.562252 2561 generic.go:356] "Generic (PLEG): container finished" podID="edc548b6-599d-469e-8362-aea863fa925d" containerID="912c671886c81a14d752fd0fdda382c9427e6f3ed2ff72af5eee8dad923ee62d" exitCode=0 Jul 22 02:22:34.562370 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:34.562320 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" event={"ID":"edc548b6-599d-469e-8362-aea863fa925d","Type":"ContainerDied","Data":"912c671886c81a14d752fd0fdda382c9427e6f3ed2ff72af5eee8dad923ee62d"} Jul 22 02:22:34.562650 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:34.562636 2561 scope.go:117] "RemoveContainer" containerID="912c671886c81a14d752fd0fdda382c9427e6f3ed2ff72af5eee8dad923ee62d" Jul 22 02:22:35.569953 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:35.569917 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"e4e31091d599d285d3f608099ffc6e69dcb01c4b561bc6c33dd85c4ab4a953dc"} Jul 22 02:22:35.570444 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:35.570411 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a8a64a0d-ebba-4dbd-a625-6e62579b1bb2","Type":"ContainerStarted","Data":"dcd9ce6ee0f4bc5b2b37cf28642ffff7fe49d802d23f6201a1fe5bbba397943d"} Jul 22 02:22:35.576515 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:35.576490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mrjvm" event={"ID":"edc548b6-599d-469e-8362-aea863fa925d","Type":"ContainerStarted","Data":"c8bf9b7f6bf34609231e8e13520b99824afdaf1e5ebe525e58704e35505f74b4"} Jul 22 02:22:35.600126 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:35.600069 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.808767499 podStartE2EDuration="9.600047233s" totalImagesPullingTime="6.766842423s" totalInitContainerRuntime="24.437311ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:22:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:35.598176952 +0000 UTC m=+114.233913782" watchObservedRunningTime="2026-07-22 02:22:35.600047233 +0000 UTC m=+114.235784041" Jul 22 02:22:36.776505 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:36.776469 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:22:41.212848 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.212801 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7598849858-lz87x" podUID="edc31bf4-5231-4d96-8f32-08204dad564e" containerName="registry" containerID="cri-o://193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64" gracePeriod=30 Jul 22 02:22:41.452217 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.452190 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:22:41.596928 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.596900 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-bound-sa-token\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.596928 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.596932 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjwn8\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-kube-api-access-sjwn8\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.597156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.596979 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-registry-certificates\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.597156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597007 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-installation-pull-secrets\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.597156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597034 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.597156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597094 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-trusted-ca\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.597156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597116 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edc31bf4-5231-4d96-8f32-08204dad564e-ca-trust-extracted\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.597156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597137 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-image-registry-private-configuration\") pod \"edc31bf4-5231-4d96-8f32-08204dad564e\" (UID: \"edc31bf4-5231-4d96-8f32-08204dad564e\") " Jul 22 02:22:41.598051 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597483 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:22:41.598051 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.597750 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:22:41.600144 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.600088 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:22:41.600461 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.600434 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:22:41.600568 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.600493 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-kube-api-access-sjwn8" (OuterVolumeSpecName: "kube-api-access-sjwn8") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "kube-api-access-sjwn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:22:41.600880 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.600852 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:22:41.600975 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.600883 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:22:41.601316 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.601289 2561 generic.go:356] "Generic (PLEG): container finished" podID="edc31bf4-5231-4d96-8f32-08204dad564e" containerID="193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64" exitCode=0 Jul 22 02:22:41.601404 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.601389 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7598849858-lz87x" Jul 22 02:22:41.601438 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.601412 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7598849858-lz87x" event={"ID":"edc31bf4-5231-4d96-8f32-08204dad564e","Type":"ContainerDied","Data":"193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64"} Jul 22 02:22:41.601511 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.601497 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7598849858-lz87x" event={"ID":"edc31bf4-5231-4d96-8f32-08204dad564e","Type":"ContainerDied","Data":"67b377417d563cc865ba4efca64ec990bc129b144deaa415337b6f39138a95d0"} Jul 22 02:22:41.601545 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.601531 2561 scope.go:117] "RemoveContainer" containerID="193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64" Jul 22 02:22:41.606857 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.606753 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc31bf4-5231-4d96-8f32-08204dad564e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "edc31bf4-5231-4d96-8f32-08204dad564e" (UID: "edc31bf4-5231-4d96-8f32-08204dad564e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:22:41.616397 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.616377 2561 scope.go:117] "RemoveContainer" containerID="193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64" Jul 22 02:22:41.616729 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:22:41.616709 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64\": container with ID starting with 193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64 not found: ID does not exist" containerID="193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64" Jul 22 02:22:41.616816 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.616742 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64"} err="failed to get container status \"193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64\": rpc error: code = NotFound desc = could not find container \"193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64\": container with ID starting with 193d5a6b3590ffb9a547154cfba85b84c5104a11ee3a72fb566de35788f45c64 not found: ID does not exist" Jul 22 02:22:41.698462 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698422 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-bound-sa-token\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698462 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698455 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjwn8\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-kube-api-access-sjwn8\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698462 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698466 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-registry-certificates\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698677 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698475 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-installation-pull-secrets\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698677 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698486 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edc31bf4-5231-4d96-8f32-08204dad564e-registry-tls\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698677 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698495 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edc31bf4-5231-4d96-8f32-08204dad564e-trusted-ca\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698677 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698502 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edc31bf4-5231-4d96-8f32-08204dad564e-ca-trust-extracted\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.698677 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.698510 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edc31bf4-5231-4d96-8f32-08204dad564e-image-registry-private-configuration\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:22:41.918474 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.918444 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7598849858-lz87x"] Jul 22 02:22:41.922454 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:41.922424 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7598849858-lz87x"] Jul 22 02:22:43.870196 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:43.870161 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edc31bf4-5231-4d96-8f32-08204dad564e" path="/var/lib/kubelet/pods/edc31bf4-5231-4d96-8f32-08204dad564e/volumes" Jul 22 02:22:48.628741 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:48.628703 2561 generic.go:356] "Generic (PLEG): container finished" podID="f7c31c9a-a2fc-4f93-a034-805ba7708a21" containerID="587f366934cf87c2a196470040ed15d64f34acc41d93b81552a5235f514834a1" exitCode=0 Jul 22 02:22:48.629125 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:48.628801 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-st5bj" event={"ID":"f7c31c9a-a2fc-4f93-a034-805ba7708a21","Type":"ContainerDied","Data":"587f366934cf87c2a196470040ed15d64f34acc41d93b81552a5235f514834a1"} Jul 22 02:22:48.629223 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:48.629204 2561 scope.go:117] "RemoveContainer" containerID="587f366934cf87c2a196470040ed15d64f34acc41d93b81552a5235f514834a1" Jul 22 02:22:49.635365 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:49.635331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-st5bj" event={"ID":"f7c31c9a-a2fc-4f93-a034-805ba7708a21","Type":"ContainerStarted","Data":"a9b19630debaa23d1f38c1aa74c559552614eeb95e2d1365647549bd142a4cc6"} Jul 22 02:22:58.670298 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:58.670257 2561 generic.go:356] "Generic (PLEG): container finished" podID="4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3" containerID="de684ae9d6d2cdd89a05c72bf552cbb80d7aaca72fbc5e5a20172303488bac89" exitCode=0 Jul 22 02:22:58.670730 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:58.670340 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" event={"ID":"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3","Type":"ContainerDied","Data":"de684ae9d6d2cdd89a05c72bf552cbb80d7aaca72fbc5e5a20172303488bac89"} Jul 22 02:22:58.670730 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:58.670685 2561 scope.go:117] "RemoveContainer" containerID="de684ae9d6d2cdd89a05c72bf552cbb80d7aaca72fbc5e5a20172303488bac89" Jul 22 02:22:59.675637 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:22:59.675596 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-2g47p" event={"ID":"4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3","Type":"ContainerStarted","Data":"baeebbe6e7aa727015d3198881fce45206e25ca88f524da7e485108a76a697b5"} Jul 22 02:23:26.776184 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:23:26.776147 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:23:26.792487 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:23:26.792459 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:23:27.797916 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:23:27.797883 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:25:41.738685 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:25:41.738654 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:25:41.745342 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:25:41.745320 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:25:41.747476 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:25:41.747450 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 02:26:00.248678 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.248644 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862"] Jul 22 02:26:00.251045 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.249011 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc31bf4-5231-4d96-8f32-08204dad564e" containerName="registry" Jul 22 02:26:00.251045 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.249024 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc31bf4-5231-4d96-8f32-08204dad564e" containerName="registry" Jul 22 02:26:00.251045 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.249070 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc31bf4-5231-4d96-8f32-08204dad564e" containerName="registry" Jul 22 02:26:00.251899 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.251883 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.255407 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.255384 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 02:26:00.255623 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.255608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 02:26:00.255715 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.255659 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 02:26:00.256150 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.256133 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 02:26:00.256469 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.256450 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z24rf\"" Jul 22 02:26:00.274345 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.274319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862"] Jul 22 02:26:00.328753 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.328714 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579a6e76-66ea-4dda-941e-66041b73a322-manifests\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.328962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.328758 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579a6e76-66ea-4dda-941e-66041b73a322-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.328962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.328837 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579a6e76-66ea-4dda-941e-66041b73a322-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.328962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.328897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579a6e76-66ea-4dda-941e-66041b73a322-tmp\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.328962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.328926 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5hqs\" (UniqueName: \"kubernetes.io/projected/579a6e76-66ea-4dda-941e-66041b73a322-kube-api-access-l5hqs\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.429432 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.429395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579a6e76-66ea-4dda-941e-66041b73a322-manifests\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.429432 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.429435 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579a6e76-66ea-4dda-941e-66041b73a322-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.429694 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.429454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579a6e76-66ea-4dda-941e-66041b73a322-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.429694 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.429477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579a6e76-66ea-4dda-941e-66041b73a322-tmp\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.429694 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.429502 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hqs\" (UniqueName: \"kubernetes.io/projected/579a6e76-66ea-4dda-941e-66041b73a322-kube-api-access-l5hqs\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.429920 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.429894 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579a6e76-66ea-4dda-941e-66041b73a322-manifests\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.432063 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.432025 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579a6e76-66ea-4dda-941e-66041b73a322-tmp\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.432172 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.432102 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579a6e76-66ea-4dda-941e-66041b73a322-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.432245 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.432181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579a6e76-66ea-4dda-941e-66041b73a322-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.442307 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.442285 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5hqs\" (UniqueName: \"kubernetes.io/projected/579a6e76-66ea-4dda-941e-66041b73a322-kube-api-access-l5hqs\") pod \"opendatahub-operator-controller-manager-5fcd77b786-z9862\" (UID: \"579a6e76-66ea-4dda-941e-66041b73a322\") " pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.561570 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.561539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:00.733310 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.733269 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862"] Jul 22 02:26:00.737429 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:26:00.737401 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579a6e76_66ea_4dda_941e_66041b73a322.slice/crio-a82728170201554dc06248c63a9825c0fd98cd5d5ec234442bbe830c7da4b94b WatchSource:0}: Error finding container a82728170201554dc06248c63a9825c0fd98cd5d5ec234442bbe830c7da4b94b: Status 404 returned error can't find the container with id a82728170201554dc06248c63a9825c0fd98cd5d5ec234442bbe830c7da4b94b Jul 22 02:26:00.739406 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:00.739387 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:26:01.351922 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:01.351853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" event={"ID":"579a6e76-66ea-4dda-941e-66041b73a322","Type":"ContainerStarted","Data":"a82728170201554dc06248c63a9825c0fd98cd5d5ec234442bbe830c7da4b94b"} Jul 22 02:26:04.366874 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:04.366833 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" event={"ID":"579a6e76-66ea-4dda-941e-66041b73a322","Type":"ContainerStarted","Data":"fa260b5bfc9c8d473ad24dbf83910ce1d1a31cbfacb4b4ae531a3b7c4985f961"} Jul 22 02:26:05.373045 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:05.373006 2561 generic.go:356] "Generic (PLEG): container finished" podID="579a6e76-66ea-4dda-941e-66041b73a322" containerID="fa260b5bfc9c8d473ad24dbf83910ce1d1a31cbfacb4b4ae531a3b7c4985f961" exitCode=0 Jul 22 02:26:05.373452 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:05.373056 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" event={"ID":"579a6e76-66ea-4dda-941e-66041b73a322","Type":"ContainerDied","Data":"fa260b5bfc9c8d473ad24dbf83910ce1d1a31cbfacb4b4ae531a3b7c4985f961"} Jul 22 02:26:06.379870 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:06.379833 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" event={"ID":"579a6e76-66ea-4dda-941e-66041b73a322","Type":"ContainerStarted","Data":"1804d01553a038068530c26f455fc66c75e6954606ecba41626ebbce02b2e07e"} Jul 22 02:26:06.380298 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:06.379908 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:06.405944 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:06.405876 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" podStartSLOduration=2.156556257 podStartE2EDuration="6.405859284s" totalImagesPullingTime="3.21441138s" totalInitContainerRuntime="1.034891647s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:06.405614887 +0000 UTC m=+325.041351692" watchObservedRunningTime="2026-07-22 02:26:06.405859284 +0000 UTC m=+325.041596088" Jul 22 02:26:17.386429 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:17.386397 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fcd77b786-z9862" Jul 22 02:26:18.736540 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.736500 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq"] Jul 22 02:26:18.745939 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.745913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.753620 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.753594 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 02:26:18.753818 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.753622 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:26:18.753818 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.753622 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 02:26:18.753818 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.753660 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 02:26:18.753995 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.753913 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ssbg9\"" Jul 22 02:26:18.754426 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.754407 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 02:26:18.774272 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.774245 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq"] Jul 22 02:26:18.883244 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.883208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e72b07b-6a13-4dff-9cfc-264d3c83819b-cert\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.883244 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.883247 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6e72b07b-6a13-4dff-9cfc-264d3c83819b-manager-config\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.883447 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.883373 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e72b07b-6a13-4dff-9cfc-264d3c83819b-metrics-cert\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.883447 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.883402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w526k\" (UniqueName: \"kubernetes.io/projected/6e72b07b-6a13-4dff-9cfc-264d3c83819b-kube-api-access-w526k\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.984732 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.984689 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6e72b07b-6a13-4dff-9cfc-264d3c83819b-manager-config\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.984944 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.984830 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e72b07b-6a13-4dff-9cfc-264d3c83819b-metrics-cert\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.984944 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.984856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w526k\" (UniqueName: \"kubernetes.io/projected/6e72b07b-6a13-4dff-9cfc-264d3c83819b-kube-api-access-w526k\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.984944 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.984888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e72b07b-6a13-4dff-9cfc-264d3c83819b-cert\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.985558 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.985533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6e72b07b-6a13-4dff-9cfc-264d3c83819b-manager-config\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.987668 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.987620 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e72b07b-6a13-4dff-9cfc-264d3c83819b-cert\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.987668 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.987620 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e72b07b-6a13-4dff-9cfc-264d3c83819b-metrics-cert\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:18.995417 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:18.995395 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w526k\" (UniqueName: \"kubernetes.io/projected/6e72b07b-6a13-4dff-9cfc-264d3c83819b-kube-api-access-w526k\") pod \"lws-controller-manager-6b78567f4c-vt2cq\" (UID: \"6e72b07b-6a13-4dff-9cfc-264d3c83819b\") " pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:19.055442 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:19.055406 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:19.206343 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:19.206309 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq"] Jul 22 02:26:19.209653 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:26:19.209623 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e72b07b_6a13_4dff_9cfc_264d3c83819b.slice/crio-722ed9d75a052afe6b841cd406c72a644cdf440046a35b33bca204d2d88dd47c WatchSource:0}: Error finding container 722ed9d75a052afe6b841cd406c72a644cdf440046a35b33bca204d2d88dd47c: Status 404 returned error can't find the container with id 722ed9d75a052afe6b841cd406c72a644cdf440046a35b33bca204d2d88dd47c Jul 22 02:26:19.430306 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:19.430269 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" event={"ID":"6e72b07b-6a13-4dff-9cfc-264d3c83819b","Type":"ContainerStarted","Data":"722ed9d75a052afe6b841cd406c72a644cdf440046a35b33bca204d2d88dd47c"} Jul 22 02:26:23.449319 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:23.449292 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" event={"ID":"6e72b07b-6a13-4dff-9cfc-264d3c83819b","Type":"ContainerStarted","Data":"fc477392800203711039e68b72998c4c3a5401fcae16849ef81b7ed0103ef537"} Jul 22 02:26:23.449650 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:23.449423 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:26:23.471639 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:23.471583 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" podStartSLOduration=1.302291767 podStartE2EDuration="5.471565113s" totalImagesPullingTime="4.169273346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:23.470611193 +0000 UTC m=+342.106347999" watchObservedRunningTime="2026-07-22 02:26:23.471565113 +0000 UTC m=+342.107301918" Jul 22 02:26:25.991478 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:25.991428 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6"] Jul 22 02:26:25.994691 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:25.994669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:25.997172 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:25.997148 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 02:26:25.997289 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:25.997170 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 02:26:25.997289 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:25.997194 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4b5x8\"" Jul 22 02:26:26.011790 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.011745 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6"] Jul 22 02:26:26.152164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.152130 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a996b46-59b1-4f27-b843-4023bd3892b2-tmp\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.152164 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.152169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4gnr\" (UniqueName: \"kubernetes.io/projected/4a996b46-59b1-4f27-b843-4023bd3892b2-kube-api-access-d4gnr\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.152386 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.152225 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a996b46-59b1-4f27-b843-4023bd3892b2-tls-certs\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.252912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.252806 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a996b46-59b1-4f27-b843-4023bd3892b2-tmp\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.252912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.252858 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4gnr\" (UniqueName: \"kubernetes.io/projected/4a996b46-59b1-4f27-b843-4023bd3892b2-kube-api-access-d4gnr\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.252912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.252900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a996b46-59b1-4f27-b843-4023bd3892b2-tls-certs\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.253199 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:26:26.253015 2561 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 22 02:26:26.253199 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:26:26.253091 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a996b46-59b1-4f27-b843-4023bd3892b2-tls-certs podName:4a996b46-59b1-4f27-b843-4023bd3892b2 nodeName:}" failed. No retries permitted until 2026-07-22 02:26:26.753070031 +0000 UTC m=+345.388806814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4a996b46-59b1-4f27-b843-4023bd3892b2-tls-certs") pod "kube-auth-proxy-79676bf5cc-lj2z6" (UID: "4a996b46-59b1-4f27-b843-4023bd3892b2") : secret "kube-auth-proxy-tls" not found Jul 22 02:26:26.255322 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.255293 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a996b46-59b1-4f27-b843-4023bd3892b2-tmp\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.262511 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.262487 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4gnr\" (UniqueName: \"kubernetes.io/projected/4a996b46-59b1-4f27-b843-4023bd3892b2-kube-api-access-d4gnr\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.756022 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.755985 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a996b46-59b1-4f27-b843-4023bd3892b2-tls-certs\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.758528 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.758500 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a996b46-59b1-4f27-b843-4023bd3892b2-tls-certs\") pod \"kube-auth-proxy-79676bf5cc-lj2z6\" (UID: \"4a996b46-59b1-4f27-b843-4023bd3892b2\") " pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:26.904234 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:26.904199 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" Jul 22 02:26:27.050854 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:27.050825 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6"] Jul 22 02:26:27.051794 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:26:27.051746 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a996b46_59b1_4f27_b843_4023bd3892b2.slice/crio-953f8324281953a68dff867d81ecb95d617d378a3c237dccf7a571c224817fa1 WatchSource:0}: Error finding container 953f8324281953a68dff867d81ecb95d617d378a3c237dccf7a571c224817fa1: Status 404 returned error can't find the container with id 953f8324281953a68dff867d81ecb95d617d378a3c237dccf7a571c224817fa1 Jul 22 02:26:27.466887 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:27.466802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" event={"ID":"4a996b46-59b1-4f27-b843-4023bd3892b2","Type":"ContainerStarted","Data":"953f8324281953a68dff867d81ecb95d617d378a3c237dccf7a571c224817fa1"} Jul 22 02:26:29.477382 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:29.477297 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" event={"ID":"4a996b46-59b1-4f27-b843-4023bd3892b2","Type":"ContainerStarted","Data":"5d92f9c6dab1b2d8bca89c2f781a33fa0bfe8f10551458e76a52536e8d01604e"} Jul 22 02:26:29.526087 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:29.526031 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79676bf5cc-lj2z6" podStartSLOduration=2.452951147 podStartE2EDuration="4.526014751s" totalImagesPullingTime="2.073063604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:29.525667784 +0000 UTC m=+348.161404589" watchObservedRunningTime="2026-07-22 02:26:29.526014751 +0000 UTC m=+348.161751556" Jul 22 02:26:34.455568 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:26:34.455539 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6b78567f4c-vt2cq" Jul 22 02:28:00.168237 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.168199 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw"] Jul 22 02:28:00.171243 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.171226 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:00.174132 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.174107 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 02:28:00.174248 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.174185 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 02:28:00.174404 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.174388 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 02:28:00.175066 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.175051 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jjr46\"" Jul 22 02:28:00.184701 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.184680 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw"] Jul 22 02:28:00.226841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.226819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8qxt\" (UniqueName: \"kubernetes.io/projected/4d183f4e-b01e-4100-a679-aa3631ffe6d5-kube-api-access-l8qxt\") pod \"dns-operator-controller-manager-864dd78c4c-s2qcw\" (UID: \"4d183f4e-b01e-4100-a679-aa3631ffe6d5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:00.327539 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.327515 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8qxt\" (UniqueName: \"kubernetes.io/projected/4d183f4e-b01e-4100-a679-aa3631ffe6d5-kube-api-access-l8qxt\") pod \"dns-operator-controller-manager-864dd78c4c-s2qcw\" (UID: \"4d183f4e-b01e-4100-a679-aa3631ffe6d5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:00.347823 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.347795 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8qxt\" (UniqueName: \"kubernetes.io/projected/4d183f4e-b01e-4100-a679-aa3631ffe6d5-kube-api-access-l8qxt\") pod \"dns-operator-controller-manager-864dd78c4c-s2qcw\" (UID: \"4d183f4e-b01e-4100-a679-aa3631ffe6d5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:00.481349 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.481286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:00.609755 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.609729 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw"] Jul 22 02:28:00.613031 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:28:00.613000 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d183f4e_b01e_4100_a679_aa3631ffe6d5.slice/crio-ed5ed3c038c7158cff9524dde5c6aad84138b4854b500d1c65475d4271d7f401 WatchSource:0}: Error finding container ed5ed3c038c7158cff9524dde5c6aad84138b4854b500d1c65475d4271d7f401: Status 404 returned error can't find the container with id ed5ed3c038c7158cff9524dde5c6aad84138b4854b500d1c65475d4271d7f401 Jul 22 02:28:00.851376 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:00.851345 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" event={"ID":"4d183f4e-b01e-4100-a679-aa3631ffe6d5","Type":"ContainerStarted","Data":"ed5ed3c038c7158cff9524dde5c6aad84138b4854b500d1c65475d4271d7f401"} Jul 22 02:28:02.865354 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:02.865314 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" event={"ID":"4d183f4e-b01e-4100-a679-aa3631ffe6d5","Type":"ContainerStarted","Data":"85f0cb7c979eaca82762a61bef8cb1815f963a0d9d31133aaf4a58367e910aca"} Jul 22 02:28:02.865734 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:02.865469 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:03.775800 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.775727 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" podStartSLOduration=2.107946918 podStartE2EDuration="3.775707007s" totalImagesPullingTime="1.667760089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:02.915227928 +0000 UTC m=+441.550964733" watchObservedRunningTime="2026-07-22 02:28:03.775707007 +0000 UTC m=+442.411443812" Jul 22 02:28:03.776627 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.776599 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp"] Jul 22 02:28:03.779813 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.779795 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.786275 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.786023 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 22 02:28:03.786585 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.786565 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gbctx\"" Jul 22 02:28:03.786681 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.786570 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 22 02:28:03.795471 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.795443 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp"] Jul 22 02:28:03.858134 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.858087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ab77e94-aab4-4d72-8015-4ca43445c14c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.858315 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.858146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ab77e94-aab4-4d72-8015-4ca43445c14c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.858315 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.858178 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnnk4\" (UniqueName: \"kubernetes.io/projected/1ab77e94-aab4-4d72-8015-4ca43445c14c-kube-api-access-xnnk4\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.958981 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.958928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ab77e94-aab4-4d72-8015-4ca43445c14c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.958981 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.958993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ab77e94-aab4-4d72-8015-4ca43445c14c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.959425 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.959014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnnk4\" (UniqueName: \"kubernetes.io/projected/1ab77e94-aab4-4d72-8015-4ca43445c14c-kube-api-access-xnnk4\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.959747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.959726 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ab77e94-aab4-4d72-8015-4ca43445c14c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.961521 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.961493 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ab77e94-aab4-4d72-8015-4ca43445c14c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:03.967487 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:03.967461 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnnk4\" (UniqueName: \"kubernetes.io/projected/1ab77e94-aab4-4d72-8015-4ca43445c14c-kube-api-access-xnnk4\") pod \"kuadrant-console-plugin-66b5774985-2l9kp\" (UID: \"1ab77e94-aab4-4d72-8015-4ca43445c14c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:04.098901 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:04.098866 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" Jul 22 02:28:04.234971 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:04.234937 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp"] Jul 22 02:28:04.239345 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:28:04.239320 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ab77e94_aab4_4d72_8015_4ca43445c14c.slice/crio-e65bf5d77320d33d50c85c78d2ff355b7a084aff0337924be34e931ea031cd7e WatchSource:0}: Error finding container e65bf5d77320d33d50c85c78d2ff355b7a084aff0337924be34e931ea031cd7e: Status 404 returned error can't find the container with id e65bf5d77320d33d50c85c78d2ff355b7a084aff0337924be34e931ea031cd7e Jul 22 02:28:04.877937 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:04.877894 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" event={"ID":"1ab77e94-aab4-4d72-8015-4ca43445c14c","Type":"ContainerStarted","Data":"e65bf5d77320d33d50c85c78d2ff355b7a084aff0337924be34e931ea031cd7e"} Jul 22 02:28:13.872709 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:13.872677 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s2qcw" Jul 22 02:28:25.984088 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:25.984046 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" event={"ID":"1ab77e94-aab4-4d72-8015-4ca43445c14c","Type":"ContainerStarted","Data":"a67a3fecf8d7da214229058a89c70b3a58945a8ffd09dc8f850a306bce88420d"} Jul 22 02:28:26.001240 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:26.001177 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l9kp" podStartSLOduration=2.253638871 podStartE2EDuration="23.001156905s" totalImagesPullingTime="20.747518034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:26.000249485 +0000 UTC m=+464.635986311" watchObservedRunningTime="2026-07-22 02:28:26.001156905 +0000 UTC m=+464.636893713" Jul 22 02:28:49.122673 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.122637 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:28:49.150257 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.150226 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:28:49.150257 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.150262 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:28:49.150437 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.150347 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.152567 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.152545 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 02:28:49.265858 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.265830 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eeba1e6f-97b2-404e-b207-ddbfb082391c-config-file\") pod \"limitador-limitador-54d6d988d5-ggsmp\" (UID: \"eeba1e6f-97b2-404e-b207-ddbfb082391c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.265988 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.265877 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvjn4\" (UniqueName: \"kubernetes.io/projected/eeba1e6f-97b2-404e-b207-ddbfb082391c-kube-api-access-vvjn4\") pod \"limitador-limitador-54d6d988d5-ggsmp\" (UID: \"eeba1e6f-97b2-404e-b207-ddbfb082391c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.367126 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.367102 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvjn4\" (UniqueName: \"kubernetes.io/projected/eeba1e6f-97b2-404e-b207-ddbfb082391c-kube-api-access-vvjn4\") pod \"limitador-limitador-54d6d988d5-ggsmp\" (UID: \"eeba1e6f-97b2-404e-b207-ddbfb082391c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.367252 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.367165 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eeba1e6f-97b2-404e-b207-ddbfb082391c-config-file\") pod \"limitador-limitador-54d6d988d5-ggsmp\" (UID: \"eeba1e6f-97b2-404e-b207-ddbfb082391c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.367860 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.367842 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eeba1e6f-97b2-404e-b207-ddbfb082391c-config-file\") pod \"limitador-limitador-54d6d988d5-ggsmp\" (UID: \"eeba1e6f-97b2-404e-b207-ddbfb082391c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.375124 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.375070 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvjn4\" (UniqueName: \"kubernetes.io/projected/eeba1e6f-97b2-404e-b207-ddbfb082391c-kube-api-access-vvjn4\") pod \"limitador-limitador-54d6d988d5-ggsmp\" (UID: \"eeba1e6f-97b2-404e-b207-ddbfb082391c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.460528 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.460504 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:49.594739 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.594708 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:28:49.596474 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:28:49.596439 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeba1e6f_97b2_404e_b207_ddbfb082391c.slice/crio-40b09b473345ef60e03816d47af3018a33a084d064571fc6f189f12dab3f0642 WatchSource:0}: Error finding container 40b09b473345ef60e03816d47af3018a33a084d064571fc6f189f12dab3f0642: Status 404 returned error can't find the container with id 40b09b473345ef60e03816d47af3018a33a084d064571fc6f189f12dab3f0642 Jul 22 02:28:49.822675 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.822648 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-fhnrz"] Jul 22 02:28:49.825704 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.825685 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:28:49.828247 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.828221 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4v575\"" Jul 22 02:28:49.832849 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.832803 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fhnrz"] Jul 22 02:28:49.970962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:49.970933 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnlm8\" (UniqueName: \"kubernetes.io/projected/fe66b394-8c6f-496a-8708-5b24b70362ba-kube-api-access-fnlm8\") pod \"authorino-75d556bd77-fhnrz\" (UID: \"fe66b394-8c6f-496a-8708-5b24b70362ba\") " pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:28:50.071864 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:50.071836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnlm8\" (UniqueName: \"kubernetes.io/projected/fe66b394-8c6f-496a-8708-5b24b70362ba-kube-api-access-fnlm8\") pod \"authorino-75d556bd77-fhnrz\" (UID: \"fe66b394-8c6f-496a-8708-5b24b70362ba\") " pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:28:50.082421 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:50.082359 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnlm8\" (UniqueName: \"kubernetes.io/projected/fe66b394-8c6f-496a-8708-5b24b70362ba-kube-api-access-fnlm8\") pod \"authorino-75d556bd77-fhnrz\" (UID: \"fe66b394-8c6f-496a-8708-5b24b70362ba\") " pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:28:50.089183 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:50.089119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" event={"ID":"eeba1e6f-97b2-404e-b207-ddbfb082391c","Type":"ContainerStarted","Data":"40b09b473345ef60e03816d47af3018a33a084d064571fc6f189f12dab3f0642"} Jul 22 02:28:50.135829 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:50.135804 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:28:50.316899 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:50.316829 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fhnrz"] Jul 22 02:28:50.320172 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:28:50.320131 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe66b394_8c6f_496a_8708_5b24b70362ba.slice/crio-03c375919f0bfc298cb44e5ba210a09229e6449cac50ee2fa64d8cd64ee3fef2 WatchSource:0}: Error finding container 03c375919f0bfc298cb44e5ba210a09229e6449cac50ee2fa64d8cd64ee3fef2: Status 404 returned error can't find the container with id 03c375919f0bfc298cb44e5ba210a09229e6449cac50ee2fa64d8cd64ee3fef2 Jul 22 02:28:51.098336 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:51.098274 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fhnrz" event={"ID":"fe66b394-8c6f-496a-8708-5b24b70362ba","Type":"ContainerStarted","Data":"03c375919f0bfc298cb44e5ba210a09229e6449cac50ee2fa64d8cd64ee3fef2"} Jul 22 02:28:55.120318 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:55.120277 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fhnrz" event={"ID":"fe66b394-8c6f-496a-8708-5b24b70362ba","Type":"ContainerStarted","Data":"858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af"} Jul 22 02:28:55.121844 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:55.121818 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" event={"ID":"eeba1e6f-97b2-404e-b207-ddbfb082391c","Type":"ContainerStarted","Data":"4504e7c6938d5c37784f6c2c10f0504976060c8b004dbf744b8d47afc0f04f89"} Jul 22 02:28:55.121958 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:55.121871 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:28:55.146022 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:55.145974 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-fhnrz" podStartSLOduration=2.155217162 podStartE2EDuration="6.145960366s" totalImagesPullingTime="3.990743204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:55.144969581 +0000 UTC m=+493.780706385" watchObservedRunningTime="2026-07-22 02:28:55.145960366 +0000 UTC m=+493.781697150" Jul 22 02:28:55.164286 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:28:55.164237 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" podStartSLOduration=1.4509425679999999 podStartE2EDuration="6.164222023s" totalImagesPullingTime="4.713279455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:55.162978402 +0000 UTC m=+493.798715208" watchObservedRunningTime="2026-07-22 02:28:55.164222023 +0000 UTC m=+493.799958829" Jul 22 02:29:06.127036 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:06.127001 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggsmp" Jul 22 02:29:33.803549 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:33.803515 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56d46ddb99-c2vxs"] Jul 22 02:29:33.811068 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:33.811038 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:33.813214 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:33.813186 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-c2vxs"] Jul 22 02:29:33.871523 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:33.871491 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-c2vxs"] Jul 22 02:29:33.871739 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:29:33.871722 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tw9dk], unattached volumes=[], failed to process volumes=[kube-api-access-tw9dk]: context canceled" pod="kuadrant-system/authorino-56d46ddb99-c2vxs" podUID="98f3dfa7-fb39-4f39-8d9f-5d12881613e7" Jul 22 02:29:33.944098 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:33.944062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw9dk\" (UniqueName: \"kubernetes.io/projected/98f3dfa7-fb39-4f39-8d9f-5d12881613e7-kube-api-access-tw9dk\") pod \"authorino-56d46ddb99-c2vxs\" (UID: \"98f3dfa7-fb39-4f39-8d9f-5d12881613e7\") " pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:34.044642 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.044608 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw9dk\" (UniqueName: \"kubernetes.io/projected/98f3dfa7-fb39-4f39-8d9f-5d12881613e7-kube-api-access-tw9dk\") pod \"authorino-56d46ddb99-c2vxs\" (UID: \"98f3dfa7-fb39-4f39-8d9f-5d12881613e7\") " pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:34.052855 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.052832 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw9dk\" (UniqueName: \"kubernetes.io/projected/98f3dfa7-fb39-4f39-8d9f-5d12881613e7-kube-api-access-tw9dk\") pod \"authorino-56d46ddb99-c2vxs\" (UID: \"98f3dfa7-fb39-4f39-8d9f-5d12881613e7\") " pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:34.291252 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.291219 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:34.297378 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.297357 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:34.347355 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.347319 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw9dk\" (UniqueName: \"kubernetes.io/projected/98f3dfa7-fb39-4f39-8d9f-5d12881613e7-kube-api-access-tw9dk\") pod \"98f3dfa7-fb39-4f39-8d9f-5d12881613e7\" (UID: \"98f3dfa7-fb39-4f39-8d9f-5d12881613e7\") " Jul 22 02:29:34.349542 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.349515 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f3dfa7-fb39-4f39-8d9f-5d12881613e7-kube-api-access-tw9dk" (OuterVolumeSpecName: "kube-api-access-tw9dk") pod "98f3dfa7-fb39-4f39-8d9f-5d12881613e7" (UID: "98f3dfa7-fb39-4f39-8d9f-5d12881613e7"). InnerVolumeSpecName "kube-api-access-tw9dk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:29:34.448545 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:34.448507 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw9dk\" (UniqueName: \"kubernetes.io/projected/98f3dfa7-fb39-4f39-8d9f-5d12881613e7-kube-api-access-tw9dk\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:29:35.020842 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.020803 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fhnrz"] Jul 22 02:29:35.021226 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.021033 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-fhnrz" podUID="fe66b394-8c6f-496a-8708-5b24b70362ba" containerName="authorino" containerID="cri-o://858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af" gracePeriod=30 Jul 22 02:29:35.271540 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.271488 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:29:35.296677 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.296641 2561 generic.go:356] "Generic (PLEG): container finished" podID="fe66b394-8c6f-496a-8708-5b24b70362ba" containerID="858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af" exitCode=0 Jul 22 02:29:35.296841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.296677 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fhnrz" Jul 22 02:29:35.296841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.296690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fhnrz" event={"ID":"fe66b394-8c6f-496a-8708-5b24b70362ba","Type":"ContainerDied","Data":"858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af"} Jul 22 02:29:35.296841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.296752 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fhnrz" event={"ID":"fe66b394-8c6f-496a-8708-5b24b70362ba","Type":"ContainerDied","Data":"03c375919f0bfc298cb44e5ba210a09229e6449cac50ee2fa64d8cd64ee3fef2"} Jul 22 02:29:35.296841 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.296790 2561 scope.go:117] "RemoveContainer" containerID="858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af" Jul 22 02:29:35.297040 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.296953 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d46ddb99-c2vxs" Jul 22 02:29:35.307138 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.307121 2561 scope.go:117] "RemoveContainer" containerID="858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af" Jul 22 02:29:35.307411 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:29:35.307392 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af\": container with ID starting with 858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af not found: ID does not exist" containerID="858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af" Jul 22 02:29:35.307467 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.307420 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af"} err="failed to get container status \"858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af\": rpc error: code = NotFound desc = could not find container \"858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af\": container with ID starting with 858051c02fcde79865751557b49aa7cdda98186905046828a1861db10075e8af not found: ID does not exist" Jul 22 02:29:35.323651 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.323627 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-c2vxs"] Jul 22 02:29:35.327815 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.327792 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56d46ddb99-c2vxs"] Jul 22 02:29:35.355547 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.355521 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnlm8\" (UniqueName: \"kubernetes.io/projected/fe66b394-8c6f-496a-8708-5b24b70362ba-kube-api-access-fnlm8\") pod \"fe66b394-8c6f-496a-8708-5b24b70362ba\" (UID: \"fe66b394-8c6f-496a-8708-5b24b70362ba\") " Jul 22 02:29:35.357866 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.357838 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe66b394-8c6f-496a-8708-5b24b70362ba-kube-api-access-fnlm8" (OuterVolumeSpecName: "kube-api-access-fnlm8") pod "fe66b394-8c6f-496a-8708-5b24b70362ba" (UID: "fe66b394-8c6f-496a-8708-5b24b70362ba"). InnerVolumeSpecName "kube-api-access-fnlm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:29:35.456549 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.456512 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnlm8\" (UniqueName: \"kubernetes.io/projected/fe66b394-8c6f-496a-8708-5b24b70362ba-kube-api-access-fnlm8\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:29:35.616353 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.616323 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fhnrz"] Jul 22 02:29:35.619682 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.619658 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fhnrz"] Jul 22 02:29:35.870875 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.870800 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f3dfa7-fb39-4f39-8d9f-5d12881613e7" path="/var/lib/kubelet/pods/98f3dfa7-fb39-4f39-8d9f-5d12881613e7/volumes" Jul 22 02:29:35.871056 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:29:35.871043 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe66b394-8c6f-496a-8708-5b24b70362ba" path="/var/lib/kubelet/pods/fe66b394-8c6f-496a-8708-5b24b70362ba/volumes" Jul 22 02:30:17.697386 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:17.697348 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:30:33.870898 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.870869 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x"] Jul 22 02:30:33.871280 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.871212 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe66b394-8c6f-496a-8708-5b24b70362ba" containerName="authorino" Jul 22 02:30:33.871280 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.871228 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe66b394-8c6f-496a-8708-5b24b70362ba" containerName="authorino" Jul 22 02:30:33.871350 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.871313 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe66b394-8c6f-496a-8708-5b24b70362ba" containerName="authorino" Jul 22 02:30:33.874474 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.874453 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:33.877205 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.877182 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 02:30:33.877205 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.877198 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 02:30:33.878146 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.878124 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 02:30:33.878487 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.878468 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dz5z5\"" Jul 22 02:30:33.881141 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.881120 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x"] Jul 22 02:30:33.945795 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.945745 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:33.945974 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.945830 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:33.945974 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.945878 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:33.945974 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.945929 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7hzn\" (UniqueName: \"kubernetes.io/projected/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-kube-api-access-r7hzn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:33.946073 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.945992 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:33.946073 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:33.946011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.046842 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.046794 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.046842 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.046848 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047060 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.046887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7hzn\" (UniqueName: \"kubernetes.io/projected/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-kube-api-access-r7hzn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047060 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.046932 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047060 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.046962 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047060 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.047002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047281 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.047258 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047356 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.047330 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.047391 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.047340 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.049459 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.049429 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.049741 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.049717 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.054990 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.054960 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7hzn\" (UniqueName: \"kubernetes.io/projected/c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e-kube-api-access-r7hzn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x\" (UID: \"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.192945 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.192852 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:34.305882 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.305851 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:30:34.335531 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.335504 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x"] Jul 22 02:30:34.337735 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:30:34.337691 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60f9ba0_84a4_4bae_9aeb_ebbdf4548c4e.slice/crio-0fde9cff4eda29d37b2eb4bc1356bc35c4d8080127b0961aecbfcd267621a3fc WatchSource:0}: Error finding container 0fde9cff4eda29d37b2eb4bc1356bc35c4d8080127b0961aecbfcd267621a3fc: Status 404 returned error can't find the container with id 0fde9cff4eda29d37b2eb4bc1356bc35c4d8080127b0961aecbfcd267621a3fc Jul 22 02:30:34.553914 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:34.553877 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" event={"ID":"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e","Type":"ContainerStarted","Data":"0fde9cff4eda29d37b2eb4bc1356bc35c4d8080127b0961aecbfcd267621a3fc"} Jul 22 02:30:35.560371 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:35.560331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" event={"ID":"c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e","Type":"ContainerStarted","Data":"559fd894b87f87bef625e8dc285c9f2d0a04f4ccecfeb48d40d1998bcd003217"} Jul 22 02:30:35.579961 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:35.579911 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" podStartSLOduration=1.5064417429999999 podStartE2EDuration="2.579895326s" totalImagesPullingTime="1.073453583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:30:35.57781252 +0000 UTC m=+594.213549325" watchObservedRunningTime="2026-07-22 02:30:35.579895326 +0000 UTC m=+594.215632131" Jul 22 02:30:41.768587 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:41.768553 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:30:41.771786 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:41.771741 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:30:44.193949 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:44.193909 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:44.194355 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:44.193978 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:44.206529 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:44.206495 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:44.614095 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:44.614063 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x" Jul 22 02:30:50.989563 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:50.989527 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:30:59.615104 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:30:59.615058 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:31:17.002307 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.002212 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt"] Jul 22 02:31:17.010927 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.010900 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.019017 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.018998 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 02:31:17.025881 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.025860 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt"] Jul 22 02:31:17.125231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.125196 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.125231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.125234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.125453 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.125272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbl8q\" (UniqueName: \"kubernetes.io/projected/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-kube-api-access-gbl8q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.125453 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.125341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.125453 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.125383 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.125453 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.125403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.226632 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.226599 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.226632 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.226636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.226862 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.226662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbl8q\" (UniqueName: \"kubernetes.io/projected/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-kube-api-access-gbl8q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.226862 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.226806 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.226862 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.226857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.227028 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.226883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.227185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.227154 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.227185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.227182 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.227334 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.227281 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.229154 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.229130 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.229292 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.229276 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.238556 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.238533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbl8q\" (UniqueName: \"kubernetes.io/projected/21622bfe-9a67-40d5-8f26-5e0bebc6cb87-kube-api-access-gbl8q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt\" (UID: \"21622bfe-9a67-40d5-8f26-5e0bebc6cb87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.321481 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.321449 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:17.463628 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:31:17.463596 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21622bfe_9a67_40d5_8f26_5e0bebc6cb87.slice/crio-50d2b3b45d3fa62e653e1fcb95ebb222899d950900db372a533ba752e2e58699 WatchSource:0}: Error finding container 50d2b3b45d3fa62e653e1fcb95ebb222899d950900db372a533ba752e2e58699: Status 404 returned error can't find the container with id 50d2b3b45d3fa62e653e1fcb95ebb222899d950900db372a533ba752e2e58699 Jul 22 02:31:17.464925 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.464895 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt"] Jul 22 02:31:17.465045 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.465031 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:31:17.742448 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.742413 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" event={"ID":"21622bfe-9a67-40d5-8f26-5e0bebc6cb87","Type":"ContainerStarted","Data":"060ecbce1e20bdde36d10851975d9dd815247af019a83901e31eca4eda7d2314"} Jul 22 02:31:17.742634 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.742467 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" event={"ID":"21622bfe-9a67-40d5-8f26-5e0bebc6cb87","Type":"ContainerStarted","Data":"50d2b3b45d3fa62e653e1fcb95ebb222899d950900db372a533ba752e2e58699"} Jul 22 02:31:17.764859 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:17.764806 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" podStartSLOduration=1.6075434560000001 podStartE2EDuration="1.76478884s" totalImagesPullingTime="157.245384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:31:17.763356193 +0000 UTC m=+636.399092998" watchObservedRunningTime="2026-07-22 02:31:17.76478884 +0000 UTC m=+636.400525637" Jul 22 02:31:18.513589 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:18.513556 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:31:27.322291 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:27.322251 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:27.322752 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:27.322313 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:27.335943 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:27.335918 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:27.798034 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:27.798008 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt" Jul 22 02:31:38.486908 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:31:38.486867 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:32:07.001129 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.001095 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz"] Jul 22 02:32:07.003538 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.003521 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.007687 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.007665 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 22 02:32:07.028446 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.028415 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz"] Jul 22 02:32:07.067528 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.067490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.067681 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.067551 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47dbe9de-d859-47eb-9caf-30d48a2483f8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.067681 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.067585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.067681 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.067638 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.067681 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.067672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgtv6\" (UniqueName: \"kubernetes.io/projected/47dbe9de-d859-47eb-9caf-30d48a2483f8-kube-api-access-hgtv6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.067962 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.067694 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.168943 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.168886 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169131 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.168963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgtv6\" (UniqueName: \"kubernetes.io/projected/47dbe9de-d859-47eb-9caf-30d48a2483f8-kube-api-access-hgtv6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169131 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169000 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169134 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47dbe9de-d859-47eb-9caf-30d48a2483f8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169231 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169214 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169371 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169395 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169384 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.169551 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.169530 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.171637 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.171611 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47dbe9de-d859-47eb-9caf-30d48a2483f8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.171801 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.171782 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47dbe9de-d859-47eb-9caf-30d48a2483f8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.181562 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.181541 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgtv6\" (UniqueName: \"kubernetes.io/projected/47dbe9de-d859-47eb-9caf-30d48a2483f8-kube-api-access-hgtv6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-npzjz\" (UID: \"47dbe9de-d859-47eb-9caf-30d48a2483f8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.328335 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.328303 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:07.468391 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.468365 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz"] Jul 22 02:32:07.470526 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:32:07.470488 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47dbe9de_d859_47eb_9caf_30d48a2483f8.slice/crio-6bb81d93db9495bb9a108b9353a6557dbb3e4d9f4751d22a68f23544f887c991 WatchSource:0}: Error finding container 6bb81d93db9495bb9a108b9353a6557dbb3e4d9f4751d22a68f23544f887c991: Status 404 returned error can't find the container with id 6bb81d93db9495bb9a108b9353a6557dbb3e4d9f4751d22a68f23544f887c991 Jul 22 02:32:07.973288 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.973203 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" event={"ID":"47dbe9de-d859-47eb-9caf-30d48a2483f8","Type":"ContainerStarted","Data":"0804e9897b030663a0edfa826a5fcd45a0a767501ee888549b1a9ff299c29f03"} Jul 22 02:32:07.973288 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.973243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" event={"ID":"47dbe9de-d859-47eb-9caf-30d48a2483f8","Type":"ContainerStarted","Data":"6bb81d93db9495bb9a108b9353a6557dbb3e4d9f4751d22a68f23544f887c991"} Jul 22 02:32:07.992945 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:07.992895 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" podStartSLOduration=1.835093893 podStartE2EDuration="1.992880232s" totalImagesPullingTime="157.786339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:32:07.991387255 +0000 UTC m=+686.627124063" watchObservedRunningTime="2026-07-22 02:32:07.992880232 +0000 UTC m=+686.628617056" Jul 22 02:32:09.603044 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:09.603010 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:32:17.328624 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:17.328580 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:17.329128 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:17.328648 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:17.341621 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:17.341593 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:32:18.027483 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:32:18.027448 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-npzjz" Jul 22 02:33:02.002579 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.002546 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dc4cf4958-zrts4"] Jul 22 02:33:02.005229 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.005212 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.008408 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.008383 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 02:33:02.008557 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.008458 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4v575\"" Jul 22 02:33:02.015360 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.015337 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc4cf4958-zrts4"] Jul 22 02:33:02.143471 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.143437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/25bc005b-d35e-4827-b1b6-1c2c65395b59-tls-cert\") pod \"authorino-7dc4cf4958-zrts4\" (UID: \"25bc005b-d35e-4827-b1b6-1c2c65395b59\") " pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.143637 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.143511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95hwl\" (UniqueName: \"kubernetes.io/projected/25bc005b-d35e-4827-b1b6-1c2c65395b59-kube-api-access-95hwl\") pod \"authorino-7dc4cf4958-zrts4\" (UID: \"25bc005b-d35e-4827-b1b6-1c2c65395b59\") " pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.244140 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.244087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95hwl\" (UniqueName: \"kubernetes.io/projected/25bc005b-d35e-4827-b1b6-1c2c65395b59-kube-api-access-95hwl\") pod \"authorino-7dc4cf4958-zrts4\" (UID: \"25bc005b-d35e-4827-b1b6-1c2c65395b59\") " pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.244327 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.244200 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/25bc005b-d35e-4827-b1b6-1c2c65395b59-tls-cert\") pod \"authorino-7dc4cf4958-zrts4\" (UID: \"25bc005b-d35e-4827-b1b6-1c2c65395b59\") " pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.246920 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.246899 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/25bc005b-d35e-4827-b1b6-1c2c65395b59-tls-cert\") pod \"authorino-7dc4cf4958-zrts4\" (UID: \"25bc005b-d35e-4827-b1b6-1c2c65395b59\") " pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.252566 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.252548 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95hwl\" (UniqueName: \"kubernetes.io/projected/25bc005b-d35e-4827-b1b6-1c2c65395b59-kube-api-access-95hwl\") pod \"authorino-7dc4cf4958-zrts4\" (UID: \"25bc005b-d35e-4827-b1b6-1c2c65395b59\") " pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.314532 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.314502 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc4cf4958-zrts4" Jul 22 02:33:02.452890 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:02.452865 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc4cf4958-zrts4"] Jul 22 02:33:02.455433 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:33:02.455403 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25bc005b_d35e_4827_b1b6_1c2c65395b59.slice/crio-71d53de039a38cf89a0a5ddac86b74ccbc2067bde3e9e268fd7617733c891dc5 WatchSource:0}: Error finding container 71d53de039a38cf89a0a5ddac86b74ccbc2067bde3e9e268fd7617733c891dc5: Status 404 returned error can't find the container with id 71d53de039a38cf89a0a5ddac86b74ccbc2067bde3e9e268fd7617733c891dc5 Jul 22 02:33:03.220416 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:03.220385 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dc4cf4958-zrts4" event={"ID":"25bc005b-d35e-4827-b1b6-1c2c65395b59","Type":"ContainerStarted","Data":"ac914d5456fad8fb74944d23cb2e7bcad7a08d3883d5ab9b77982ffb7ba290b4"} Jul 22 02:33:03.220416 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:03.220424 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dc4cf4958-zrts4" event={"ID":"25bc005b-d35e-4827-b1b6-1c2c65395b59","Type":"ContainerStarted","Data":"71d53de039a38cf89a0a5ddac86b74ccbc2067bde3e9e268fd7617733c891dc5"} Jul 22 02:33:03.242847 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:03.242789 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7dc4cf4958-zrts4" podStartSLOduration=1.927316286 podStartE2EDuration="2.242744533s" totalImagesPullingTime="315.428247ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:33:03.241512822 +0000 UTC m=+741.877249628" watchObservedRunningTime="2026-07-22 02:33:03.242744533 +0000 UTC m=+741.878481339" Jul 22 02:33:33.009747 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:33.009705 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:33:42.294120 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:42.294078 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:33:53.289155 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:33:53.289120 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:35:34.591826 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:35:34.591781 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:35:41.799914 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:35:41.799873 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:35:41.801334 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:35:41.801309 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:35:46.488342 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:35:46.488301 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:36:21.401269 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:36:21.401233 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:36:31.102870 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:36:31.102831 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:36:39.805342 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:36:39.805308 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:36:48.694059 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:36:48.694020 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:36:58.907536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:36:58.907502 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:37:08.141201 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:37:08.141109 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:37:18.599904 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:37:18.599871 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:37:28.493680 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:37:28.493641 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:37:50.609187 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:37:50.609152 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:38:02.193437 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:38:02.193398 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:38:26.902071 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:38:26.902030 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:38:37.297568 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:38:37.297534 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:38:53.293798 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:38:53.293749 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:39:02.407753 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:39:02.407717 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:39:11.703180 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:39:11.703141 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:39:20.892424 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:39:20.892378 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:39:29.816517 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:39:29.816484 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:40:41.836880 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:40:41.836855 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:40:41.838679 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:40:41.838651 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:41:19.686940 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:41:19.686899 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:41:29.162205 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:41:29.162159 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:41:42.002390 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:41:42.002354 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:41:55.701956 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:41:55.701908 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:42:05.395551 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:42:05.395502 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:42:14.411445 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:42:14.411408 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:42:25.493557 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:42:25.493517 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:42:34.783390 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:42:34.783352 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:42:46.786865 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:42:46.786832 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:42:55.204469 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:42:55.204435 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:43:05.693795 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:05.693671 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:43:27.401195 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:27.401153 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:43:37.089239 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:37.089207 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:43:46.875156 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.875127 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2"] Jul 22 02:43:46.877918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.877902 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:46.880717 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.880695 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-bec2a4d0-kserve-self-signed-certs\"" Jul 22 02:43:46.893171 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.893144 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2"] Jul 22 02:43:46.946992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.946964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-dshm\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:46.946992 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.946995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-model-cache\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:46.947196 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.947076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckn8h\" (UniqueName: \"kubernetes.io/projected/b3421754-2f7a-4fbf-ac7f-9e76806a5191-kube-api-access-ckn8h\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:46.947196 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.947146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tls-certs\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:46.947196 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.947176 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-home\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:46.947315 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:46.947201 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tmp-dir\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048044 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tls-certs\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048044 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-home\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tmp-dir\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048146 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-dshm\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-model-cache\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048312 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckn8h\" (UniqueName: \"kubernetes.io/projected/b3421754-2f7a-4fbf-ac7f-9e76806a5191-kube-api-access-ckn8h\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048621 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048590 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tmp-dir\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048716 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048623 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-model-cache\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.048792 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.048700 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-home\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.050558 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.050534 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-dshm\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.050781 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.050747 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tls-certs\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.061143 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.061123 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckn8h\" (UniqueName: \"kubernetes.io/projected/b3421754-2f7a-4fbf-ac7f-9e76806a5191-kube-api-access-ckn8h\") pod \"e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.187552 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.187468 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:47.328543 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:47.328516 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2"] Jul 22 02:43:47.330607 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:43:47.330579 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3421754_2f7a_4fbf_ac7f_9e76806a5191.slice/crio-4d760d7d8b0cc91286f215a9982608bd1a7e341ae8a3c10933756ae212ee1007 WatchSource:0}: Error finding container 4d760d7d8b0cc91286f215a9982608bd1a7e341ae8a3c10933756ae212ee1007: Status 404 returned error can't find the container with id 4d760d7d8b0cc91286f215a9982608bd1a7e341ae8a3c10933756ae212ee1007 Jul 22 02:43:48.094180 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:48.094140 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:43:48.176052 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:48.176011 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" event={"ID":"b3421754-2f7a-4fbf-ac7f-9e76806a5191","Type":"ContainerStarted","Data":"6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d"} Jul 22 02:43:48.176052 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:48.176057 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" event={"ID":"b3421754-2f7a-4fbf-ac7f-9e76806a5191","Type":"ContainerStarted","Data":"4d760d7d8b0cc91286f215a9982608bd1a7e341ae8a3c10933756ae212ee1007"} Jul 22 02:43:57.188639 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:57.188587 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:57.189116 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:57.188729 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:57.201477 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:57.201456 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:43:57.226278 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:57.226232 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" podStartSLOduration=11.226218103 podStartE2EDuration="11.226218103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:43:48.223905584 +0000 UTC m=+1386.859642388" watchObservedRunningTime="2026-07-22 02:43:57.226218103 +0000 UTC m=+1395.861954908" Jul 22 02:43:57.233147 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:43:57.233124 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:44:03.214352 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:03.214314 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:44:39.072456 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:39.072424 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2"] Jul 22 02:44:41.203422 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:41.203386 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:44:54.092912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.092863 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" podUID="b3421754-2f7a-4fbf-ac7f-9e76806a5191" containerName="main" containerID="cri-o://6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d" gracePeriod=45 Jul 22 02:44:54.342896 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.342868 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:44:54.439676 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439591 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-home\") pod \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " Jul 22 02:44:54.439879 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439680 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-model-cache\") pod \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " Jul 22 02:44:54.439879 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439709 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-dshm\") pod \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " Jul 22 02:44:54.439879 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439734 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tmp-dir\") pod \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " Jul 22 02:44:54.439879 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439791 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tls-certs\") pod \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " Jul 22 02:44:54.439879 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439832 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckn8h\" (UniqueName: \"kubernetes.io/projected/b3421754-2f7a-4fbf-ac7f-9e76806a5191-kube-api-access-ckn8h\") pod \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\" (UID: \"b3421754-2f7a-4fbf-ac7f-9e76806a5191\") " Jul 22 02:44:54.440137 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439932 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-home" (OuterVolumeSpecName: "home") pod "b3421754-2f7a-4fbf-ac7f-9e76806a5191" (UID: "b3421754-2f7a-4fbf-ac7f-9e76806a5191"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:44:54.440137 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.439968 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-model-cache" (OuterVolumeSpecName: "model-cache") pod "b3421754-2f7a-4fbf-ac7f-9e76806a5191" (UID: "b3421754-2f7a-4fbf-ac7f-9e76806a5191"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:44:54.440137 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.440033 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b3421754-2f7a-4fbf-ac7f-9e76806a5191" (UID: "b3421754-2f7a-4fbf-ac7f-9e76806a5191"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:44:54.440303 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.440166 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tmp-dir\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:44:54.440303 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.440179 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-home\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:44:54.440303 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.440188 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-model-cache\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:44:54.442189 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.442156 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-dshm" (OuterVolumeSpecName: "dshm") pod "b3421754-2f7a-4fbf-ac7f-9e76806a5191" (UID: "b3421754-2f7a-4fbf-ac7f-9e76806a5191"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:44:54.442307 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.442240 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3421754-2f7a-4fbf-ac7f-9e76806a5191-kube-api-access-ckn8h" (OuterVolumeSpecName: "kube-api-access-ckn8h") pod "b3421754-2f7a-4fbf-ac7f-9e76806a5191" (UID: "b3421754-2f7a-4fbf-ac7f-9e76806a5191"). InnerVolumeSpecName "kube-api-access-ckn8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:44:54.442307 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.442258 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b3421754-2f7a-4fbf-ac7f-9e76806a5191" (UID: "b3421754-2f7a-4fbf-ac7f-9e76806a5191"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:44:54.485483 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.485447 2561 generic.go:356] "Generic (PLEG): container finished" podID="b3421754-2f7a-4fbf-ac7f-9e76806a5191" containerID="6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d" exitCode=0 Jul 22 02:44:54.485634 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.485493 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" Jul 22 02:44:54.485634 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.485520 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" event={"ID":"b3421754-2f7a-4fbf-ac7f-9e76806a5191","Type":"ContainerDied","Data":"6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d"} Jul 22 02:44:54.485634 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.485559 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2" event={"ID":"b3421754-2f7a-4fbf-ac7f-9e76806a5191","Type":"ContainerDied","Data":"4d760d7d8b0cc91286f215a9982608bd1a7e341ae8a3c10933756ae212ee1007"} Jul 22 02:44:54.485634 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.485579 2561 scope.go:117] "RemoveContainer" containerID="6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d" Jul 22 02:44:54.494543 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.494527 2561 scope.go:117] "RemoveContainer" containerID="6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d" Jul 22 02:44:54.494811 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:44:54.494792 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d\": container with ID starting with 6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d not found: ID does not exist" containerID="6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d" Jul 22 02:44:54.494863 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.494817 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d"} err="failed to get container status \"6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d\": rpc error: code = NotFound desc = could not find container \"6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d\": container with ID starting with 6da143d1cd6b602339081923faacae1b26a1631873cc80c87d2be79429b8959d not found: ID does not exist" Jul 22 02:44:54.508970 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.508946 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2"] Jul 22 02:44:54.514350 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.514330 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-bec2a4d0-kserve-7cfdd8f647-pklw2"] Jul 22 02:44:54.541078 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.541052 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3421754-2f7a-4fbf-ac7f-9e76806a5191-tls-certs\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:44:54.541078 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.541075 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckn8h\" (UniqueName: \"kubernetes.io/projected/b3421754-2f7a-4fbf-ac7f-9e76806a5191-kube-api-access-ckn8h\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:44:54.541239 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:54.541086 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3421754-2f7a-4fbf-ac7f-9e76806a5191-dshm\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:44:55.002977 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:55.002941 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:44:55.871405 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:44:55.871371 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3421754-2f7a-4fbf-ac7f-9e76806a5191" path="/var/lib/kubelet/pods/b3421754-2f7a-4fbf-ac7f-9e76806a5191/volumes" Jul 22 02:45:04.428594 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:04.428551 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:45:16.885853 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:16.885817 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:45:28.407479 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:28.407436 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:45:37.607466 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:37.607428 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:45:41.864680 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:41.864646 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:45:41.867603 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:41.867579 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:45:47.793079 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:47.793046 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:45:56.688048 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:45:56.688013 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:46:06.194754 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:46:06.194679 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:46:19.201037 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:46:19.201000 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:46:28.199975 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:46:28.199940 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:46:37.505006 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:46:37.504967 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:46:49.291102 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:46:49.291065 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:47:01.813623 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:47:01.813589 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:47:09.801361 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:47:09.801321 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:47:21.311014 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:47:21.310973 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:47:33.034225 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:47:33.034182 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:47:43.495522 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:47:43.495489 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:47:56.298229 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:47:56.298190 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:48:05.936418 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:48:05.936384 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:48:15.501321 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:48:15.501286 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:48:25.286419 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:48:25.286380 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:48:58.486480 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:48:58.486443 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:49:08.986005 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:49:08.985921 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:49:42.185931 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:49:42.185889 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:49:53.090878 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:49:53.090840 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:50:26.299946 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:50:26.299910 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:50:41.185796 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:50:41.185745 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:50:41.894756 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:50:41.894723 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:50:41.897459 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:50:41.897436 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:51:25.687251 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:51:25.687215 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:52:54.126918 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:52:54.126878 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:53:06.303044 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:53:06.303000 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:53:18.414359 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:53:18.414327 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:53:37.989972 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:53:37.989933 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:54:15.217669 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:54:15.217632 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:54:31.492063 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:54:31.492028 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:54:42.888832 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:54:42.888804 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:55:00.889415 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:55:00.889380 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:55:11.346457 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:55:11.346378 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:55:41.922934 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:55:41.922907 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:55:41.926224 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:55:41.926203 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 02:56:08.890435 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:56:08.890397 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:56:18.500516 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:56:18.500481 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:57:09.712513 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:09.712480 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:57:20.092333 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:20.092295 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:57:31.073158 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.073121 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5"] Jul 22 02:57:31.073556 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.073483 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3421754-2f7a-4fbf-ac7f-9e76806a5191" containerName="main" Jul 22 02:57:31.073556 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.073495 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3421754-2f7a-4fbf-ac7f-9e76806a5191" containerName="main" Jul 22 02:57:31.073556 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.073554 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3421754-2f7a-4fbf-ac7f-9e76806a5191" containerName="main" Jul 22 02:57:31.076459 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.076443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.078932 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.078882 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-187dc9\"/\"openshift-service-ca.crt\"" Jul 22 02:57:31.079808 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.079786 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-187dc9\"/\"kube-root-ca.crt\"" Jul 22 02:57:31.079928 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.079911 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-187dc9\"/\"default-dockercfg-lgqgz\"" Jul 22 02:57:31.080004 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.079941 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-187dc9\"/\"test-model-187dc9-kserve-self-signed-certs\"" Jul 22 02:57:31.086660 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.086638 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5"] Jul 22 02:57:31.184632 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.184595 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-dshm\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.184853 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.184638 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqhrj\" (UniqueName: \"kubernetes.io/projected/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-kube-api-access-pqhrj\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.184853 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.184680 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-home\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.184853 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.184702 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tls-certs\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.184853 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.184798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-model-cache\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.184853 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.184813 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tmp-dir\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286242 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286212 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-model-cache\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286242 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tmp-dir\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286475 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-dshm\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286475 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqhrj\" (UniqueName: \"kubernetes.io/projected/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-kube-api-access-pqhrj\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286475 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-home\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286475 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tls-certs\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286717 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286686 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tmp-dir\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286953 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286812 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-home\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.286953 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.286814 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-model-cache\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.288819 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.288799 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-dshm\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.289060 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.289041 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tls-certs\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.295028 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.295006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqhrj\" (UniqueName: \"kubernetes.io/projected/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-kube-api-access-pqhrj\") pod \"test-model-187dc9-kserve-659484d89c-j2jj5\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.387145 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.387052 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:31.520714 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.520686 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5"] Jul 22 02:57:31.522804 ip-10-0-134-165 kubenswrapper[2561]: W0722 02:57:31.522759 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a962cea_2f5c_4f23_9fb0_2ca0d3f80a5f.slice/crio-8dd245a3a86eba6bcb5e790ca7d3fab23074d6527c83ae886fb24e48a29f1407 WatchSource:0}: Error finding container 8dd245a3a86eba6bcb5e790ca7d3fab23074d6527c83ae886fb24e48a29f1407: Status 404 returned error can't find the container with id 8dd245a3a86eba6bcb5e790ca7d3fab23074d6527c83ae886fb24e48a29f1407 Jul 22 02:57:31.952832 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.952798 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" event={"ID":"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f","Type":"ContainerStarted","Data":"741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde"} Jul 22 02:57:31.953192 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.952838 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" event={"ID":"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f","Type":"ContainerStarted","Data":"8dd245a3a86eba6bcb5e790ca7d3fab23074d6527c83ae886fb24e48a29f1407"} Jul 22 02:57:31.972772 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:31.972730 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" podStartSLOduration=0.972716434 podStartE2EDuration="972.716434ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:57:31.969920488 +0000 UTC m=+2210.605657292" watchObservedRunningTime="2026-07-22 02:57:31.972716434 +0000 UTC m=+2210.608453239" Jul 22 02:57:35.893243 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:35.893209 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:57:41.387906 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:41.387872 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:41.388479 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:41.387919 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:41.400814 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:41.400762 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:42.006616 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:42.006590 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:57:54.190568 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:57:54.190527 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:58:26.858012 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:26.857980 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5"] Jul 22 02:58:32.089613 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:32.089577 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:58:41.878436 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:41.878377 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" podUID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" containerName="main" containerID="cri-o://741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde" gracePeriod=45 Jul 22 02:58:41.996126 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:41.996093 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" podUID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.34:8000/ready\": dial tcp 10.132.0.34:8000: connect: connection refused" Jul 22 02:58:42.135061 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.134993 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:58:42.216353 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216313 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqhrj\" (UniqueName: \"kubernetes.io/projected/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-kube-api-access-pqhrj\") pod \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " Jul 22 02:58:42.216536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216368 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-model-cache\") pod \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " Jul 22 02:58:42.216536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216407 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-home\") pod \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " Jul 22 02:58:42.216536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216453 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-dshm\") pod \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " Jul 22 02:58:42.216536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216485 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tls-certs\") pod \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " Jul 22 02:58:42.216536 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216524 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tmp-dir\") pod \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\" (UID: \"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f\") " Jul 22 02:58:42.216912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216671 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" (UID: "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.216912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216673 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-home" (OuterVolumeSpecName: "home") pod "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" (UID: "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.216912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216873 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" (UID: "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.216912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216895 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-model-cache\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.216912 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.216911 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-home\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.218858 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.218832 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" (UID: "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:58:42.218942 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.218912 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-dshm" (OuterVolumeSpecName: "dshm") pod "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" (UID: "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:58:42.218942 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.218919 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-kube-api-access-pqhrj" (OuterVolumeSpecName: "kube-api-access-pqhrj") pod "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" (UID: "9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f"). InnerVolumeSpecName "kube-api-access-pqhrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:42.266248 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.266213 2561 generic.go:356] "Generic (PLEG): container finished" podID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" containerID="741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde" exitCode=0 Jul 22 02:58:42.266449 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.266262 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" Jul 22 02:58:42.266449 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.266310 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" event={"ID":"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f","Type":"ContainerDied","Data":"741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde"} Jul 22 02:58:42.266449 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.266364 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5" event={"ID":"9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f","Type":"ContainerDied","Data":"8dd245a3a86eba6bcb5e790ca7d3fab23074d6527c83ae886fb24e48a29f1407"} Jul 22 02:58:42.266449 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.266390 2561 scope.go:117] "RemoveContainer" containerID="741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde" Jul 22 02:58:42.275485 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.275470 2561 scope.go:117] "RemoveContainer" containerID="741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde" Jul 22 02:58:42.275814 ip-10-0-134-165 kubenswrapper[2561]: E0722 02:58:42.275793 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde\": container with ID starting with 741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde not found: ID does not exist" containerID="741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde" Jul 22 02:58:42.275935 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.275823 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde"} err="failed to get container status \"741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde\": rpc error: code = NotFound desc = could not find container \"741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde\": container with ID starting with 741a88a33b75d30c1ee033cba7cd86da23d5e906aa017513da618bf81d32adde not found: ID does not exist" Jul 22 02:58:42.305039 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.305001 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5"] Jul 22 02:58:42.309877 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.309850 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-187dc9/test-model-187dc9-kserve-659484d89c-j2jj5"] Jul 22 02:58:42.318046 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.318021 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-dshm\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.318046 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.318046 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tls-certs\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.318185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.318083 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-tmp-dir\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:58:42.318185 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:42.318108 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqhrj\" (UniqueName: \"kubernetes.io/projected/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f-kube-api-access-pqhrj\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 02:58:43.870412 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:43.870372 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" path="/var/lib/kubelet/pods/9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f/volumes" Jul 22 02:58:47.479899 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:58:47.479865 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:59:03.591831 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:59:03.591794 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:59:21.716377 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:59:21.716342 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:59:31.092003 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:59:31.091968 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 02:59:59.688961 ip-10-0-134-165 kubenswrapper[2561]: I0722 02:59:59.688918 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:00:25.196108 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:00:25.196077 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:00:41.950709 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:00:41.950677 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 03:00:41.955041 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:00:41.955019 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 03:00:43.505044 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:00:43.505008 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:00:56.186909 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:00:56.186870 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:01:29.481520 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.481489 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc"] Jul 22 03:01:29.481947 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.481908 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" containerName="main" Jul 22 03:01:29.481947 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.481922 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" containerName="main" Jul 22 03:01:29.482019 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.481979 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a962cea-2f5c-4f23-9fb0-2ca0d3f80a5f" containerName="main" Jul 22 03:01:29.484731 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.484715 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.494667 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.494645 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d8076c\"/\"openshift-service-ca.crt\"" Jul 22 03:01:29.495437 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.495416 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d8076c\"/\"default-dockercfg-9psq7\"" Jul 22 03:01:29.495555 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.495480 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d8076c\"/\"kube-root-ca.crt\"" Jul 22 03:01:29.495619 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.495566 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d8076c\"/\"auth-test-model-d8076c-kserve-self-signed-certs\"" Jul 22 03:01:29.501415 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.501395 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc"] Jul 22 03:01:29.612427 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.612400 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-model-cache\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.612427 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.612428 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tmp-dir\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.612635 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.612455 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tls-certs\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.612635 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.612537 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-home\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.612635 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.612566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-dshm\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.612747 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.612657 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7z9w\" (UniqueName: \"kubernetes.io/projected/e6be9a02-40cf-4148-9b19-0167b9c5a58e-kube-api-access-t7z9w\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713353 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713326 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-model-cache\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713353 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tmp-dir\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713528 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713377 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tls-certs\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713528 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713403 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-home\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713528 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-dshm\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713528 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713476 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7z9w\" (UniqueName: \"kubernetes.io/projected/e6be9a02-40cf-4148-9b19-0167b9c5a58e-kube-api-access-t7z9w\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713813 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713791 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-model-cache\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713883 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713826 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-home\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.713883 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.713848 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tmp-dir\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.715869 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.715851 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-dshm\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.716030 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.716013 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tls-certs\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.722338 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.722319 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7z9w\" (UniqueName: \"kubernetes.io/projected/e6be9a02-40cf-4148-9b19-0167b9c5a58e-kube-api-access-t7z9w\") pod \"auth-test-model-d8076c-kserve-f958ff5d7-bmrjc\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.793863 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.793835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:29.928818 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:29.928796 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc"] Jul 22 03:01:29.931309 ip-10-0-134-165 kubenswrapper[2561]: W0722 03:01:29.931279 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6be9a02_40cf_4148_9b19_0167b9c5a58e.slice/crio-36610fcb5a87cfdd68629cc0738c3db7471ed97ab8e3c0fef13e0795c6972eca WatchSource:0}: Error finding container 36610fcb5a87cfdd68629cc0738c3db7471ed97ab8e3c0fef13e0795c6972eca: Status 404 returned error can't find the container with id 36610fcb5a87cfdd68629cc0738c3db7471ed97ab8e3c0fef13e0795c6972eca Jul 22 03:01:30.026181 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:30.026148 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" event={"ID":"e6be9a02-40cf-4148-9b19-0167b9c5a58e","Type":"ContainerStarted","Data":"94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7"} Jul 22 03:01:30.026340 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:30.026191 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" event={"ID":"e6be9a02-40cf-4148-9b19-0167b9c5a58e","Type":"ContainerStarted","Data":"36610fcb5a87cfdd68629cc0738c3db7471ed97ab8e3c0fef13e0795c6972eca"} Jul 22 03:01:30.046641 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:30.046554 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" podStartSLOduration=1.046540891 podStartE2EDuration="1.046540891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:01:30.045056268 +0000 UTC m=+2448.680793079" watchObservedRunningTime="2026-07-22 03:01:30.046540891 +0000 UTC m=+2448.682277695" Jul 22 03:01:33.102045 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:33.102013 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:01:39.794362 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:39.794325 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:39.794752 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:39.794409 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:39.807448 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:39.807423 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:01:40.081886 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:01:40.081810 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:02:01.503071 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:02:01.503034 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:02:15.291978 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:02:15.291941 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:02:33.689178 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:02:33.689078 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:02:48.189436 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:02:48.189401 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:03:06.697581 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:03:06.697546 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:03:24.693464 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:03:24.693423 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:03:42.084739 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:03:42.084701 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:03:56.659021 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:03:56.658977 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc"] Jul 22 03:03:59.993892 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:03:59.993846 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:04:11.679221 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:11.679172 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" podUID="e6be9a02-40cf-4148-9b19-0167b9c5a58e" containerName="main" containerID="cri-o://94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7" gracePeriod=45 Jul 22 03:04:11.930996 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:11.930929 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:04:12.063394 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063358 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-model-cache\") pod \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " Jul 22 03:04:12.063596 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063422 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tls-certs\") pod \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " Jul 22 03:04:12.063596 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063438 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-dshm\") pod \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " Jul 22 03:04:12.063596 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063462 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-home\") pod \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " Jul 22 03:04:12.063596 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063481 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tmp-dir\") pod \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " Jul 22 03:04:12.063596 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063503 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7z9w\" (UniqueName: \"kubernetes.io/projected/e6be9a02-40cf-4148-9b19-0167b9c5a58e-kube-api-access-t7z9w\") pod \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\" (UID: \"e6be9a02-40cf-4148-9b19-0167b9c5a58e\") " Jul 22 03:04:12.063884 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063654 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6be9a02-40cf-4148-9b19-0167b9c5a58e" (UID: "e6be9a02-40cf-4148-9b19-0167b9c5a58e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:04:12.063884 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063815 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-model-cache\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:04:12.063884 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063813 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6be9a02-40cf-4148-9b19-0167b9c5a58e" (UID: "e6be9a02-40cf-4148-9b19-0167b9c5a58e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:04:12.063884 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.063830 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-home" (OuterVolumeSpecName: "home") pod "e6be9a02-40cf-4148-9b19-0167b9c5a58e" (UID: "e6be9a02-40cf-4148-9b19-0167b9c5a58e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:04:12.065875 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.065853 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6be9a02-40cf-4148-9b19-0167b9c5a58e" (UID: "e6be9a02-40cf-4148-9b19-0167b9c5a58e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:04:12.065958 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.065936 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6be9a02-40cf-4148-9b19-0167b9c5a58e-kube-api-access-t7z9w" (OuterVolumeSpecName: "kube-api-access-t7z9w") pod "e6be9a02-40cf-4148-9b19-0167b9c5a58e" (UID: "e6be9a02-40cf-4148-9b19-0167b9c5a58e"). InnerVolumeSpecName "kube-api-access-t7z9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:04:12.065996 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.065944 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-dshm" (OuterVolumeSpecName: "dshm") pod "e6be9a02-40cf-4148-9b19-0167b9c5a58e" (UID: "e6be9a02-40cf-4148-9b19-0167b9c5a58e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:04:12.164602 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.164562 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tls-certs\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:04:12.164602 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.164595 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-dshm\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:04:12.164602 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.164603 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-home\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:04:12.164602 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.164610 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6be9a02-40cf-4148-9b19-0167b9c5a58e-tmp-dir\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:04:12.164939 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.164619 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7z9w\" (UniqueName: \"kubernetes.io/projected/e6be9a02-40cf-4148-9b19-0167b9c5a58e-kube-api-access-t7z9w\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:04:12.589461 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.589424 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:04:12.780968 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.780932 2561 generic.go:356] "Generic (PLEG): container finished" podID="e6be9a02-40cf-4148-9b19-0167b9c5a58e" containerID="94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7" exitCode=0 Jul 22 03:04:12.780968 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.780961 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" Jul 22 03:04:12.781518 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.780996 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" event={"ID":"e6be9a02-40cf-4148-9b19-0167b9c5a58e","Type":"ContainerDied","Data":"94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7"} Jul 22 03:04:12.781518 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.781044 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc" event={"ID":"e6be9a02-40cf-4148-9b19-0167b9c5a58e","Type":"ContainerDied","Data":"36610fcb5a87cfdd68629cc0738c3db7471ed97ab8e3c0fef13e0795c6972eca"} Jul 22 03:04:12.781518 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.781064 2561 scope.go:117] "RemoveContainer" containerID="94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7" Jul 22 03:04:12.790881 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.790859 2561 scope.go:117] "RemoveContainer" containerID="94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7" Jul 22 03:04:12.791880 ip-10-0-134-165 kubenswrapper[2561]: E0722 03:04:12.791849 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7\": container with ID starting with 94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7 not found: ID does not exist" containerID="94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7" Jul 22 03:04:12.792000 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.791889 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7"} err="failed to get container status \"94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7\": rpc error: code = NotFound desc = could not find container \"94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7\": container with ID starting with 94bde3b070f40e15bb8d60067c6d05d69482b9aa743e279529ba0bde2ae744b7 not found: ID does not exist" Jul 22 03:04:12.802997 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.802972 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc"] Jul 22 03:04:12.809036 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:12.809009 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d8076c/auth-test-model-d8076c-kserve-f958ff5d7-bmrjc"] Jul 22 03:04:13.870667 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:13.870629 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6be9a02-40cf-4148-9b19-0167b9c5a58e" path="/var/lib/kubelet/pods/e6be9a02-40cf-4148-9b19-0167b9c5a58e/volumes" Jul 22 03:04:38.586085 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.586053 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx"] Jul 22 03:04:38.586457 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.586401 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6be9a02-40cf-4148-9b19-0167b9c5a58e" containerName="main" Jul 22 03:04:38.586457 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.586412 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6be9a02-40cf-4148-9b19-0167b9c5a58e" containerName="main" Jul 22 03:04:38.586528 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.586468 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6be9a02-40cf-4148-9b19-0167b9c5a58e" containerName="main" Jul 22 03:04:38.589335 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.589318 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.592895 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.592868 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"sub-test-model-cf82b3-kserve-self-signed-certs\"" Jul 22 03:04:38.592895 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.592883 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"kube-root-ca.crt\"" Jul 22 03:04:38.593090 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.592892 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"default-dockercfg-xzbgs\"" Jul 22 03:04:38.593090 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.592868 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"openshift-service-ca.crt\"" Jul 22 03:04:38.602962 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.602939 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx"] Jul 22 03:04:38.699191 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.699160 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-dshm\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.699338 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.699209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-home\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.699338 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.699233 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-model-cache\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.699338 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.699275 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tls-certs\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.699338 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.699310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tmp-dir\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.699483 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.699374 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdkmq\" (UniqueName: \"kubernetes.io/projected/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-kube-api-access-jdkmq\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.800589 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.800560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tls-certs\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.800709 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.800599 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tmp-dir\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.800709 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.800625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdkmq\" (UniqueName: \"kubernetes.io/projected/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-kube-api-access-jdkmq\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.800709 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.800673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-dshm\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.800903 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.800829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-home\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.800903 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.800864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-model-cache\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.801088 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.801062 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tmp-dir\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.801177 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.801158 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-home\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.801237 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.801179 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-model-cache\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.803092 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.803072 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-dshm\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.803316 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.803299 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tls-certs\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.809123 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.809099 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdkmq\" (UniqueName: \"kubernetes.io/projected/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-kube-api-access-jdkmq\") pod \"sub-test-model-cf82b3-kserve-78569ddc-5bwdx\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:38.898374 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:38.898323 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:39.027870 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:39.027848 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx"] Jul 22 03:04:39.030283 ip-10-0-134-165 kubenswrapper[2561]: W0722 03:04:39.030241 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39199532_0c15_4a91_a7ef_e9d2d91fb7ab.slice/crio-9a9febc28c0b8cd37e1094c314547535655738542b43c1cfb01d7bde5d9b102e WatchSource:0}: Error finding container 9a9febc28c0b8cd37e1094c314547535655738542b43c1cfb01d7bde5d9b102e: Status 404 returned error can't find the container with id 9a9febc28c0b8cd37e1094c314547535655738542b43c1cfb01d7bde5d9b102e Jul 22 03:04:39.906875 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:39.906840 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" event={"ID":"39199532-0c15-4a91-a7ef-e9d2d91fb7ab","Type":"ContainerStarted","Data":"1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093"} Jul 22 03:04:39.907265 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:39.906883 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" event={"ID":"39199532-0c15-4a91-a7ef-e9d2d91fb7ab","Type":"ContainerStarted","Data":"9a9febc28c0b8cd37e1094c314547535655738542b43c1cfb01d7bde5d9b102e"} Jul 22 03:04:39.929267 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:39.929214 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" podStartSLOduration=1.929197989 podStartE2EDuration="1.929197989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:04:39.928082974 +0000 UTC m=+2638.563819779" watchObservedRunningTime="2026-07-22 03:04:39.929197989 +0000 UTC m=+2638.564934793" Jul 22 03:04:42.679694 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:42.679657 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:04:48.898611 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:48.898574 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:48.899136 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:48.898626 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:48.910857 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:48.910829 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:04:48.959154 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:04:48.959125 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:05:03.791700 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:03.791664 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:05:24.283450 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:24.283418 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx"] Jul 22 03:05:28.493232 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:28.493194 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:05:39.304512 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.304467 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" podUID="39199532-0c15-4a91-a7ef-e9d2d91fb7ab" containerName="main" containerID="cri-o://1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093" gracePeriod=45 Jul 22 03:05:39.341747 ip-10-0-134-165 kubenswrapper[2561]: E0722 03:05:39.341711 2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39199532_0c15_4a91_a7ef_e9d2d91fb7ab.slice/crio-conmon-1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39199532_0c15_4a91_a7ef_e9d2d91fb7ab.slice/crio-1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093.scope\": RecentStats: unable to find data in memory cache]" Jul 22 03:05:39.561895 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.561835 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:05:39.624498 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624467 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-model-cache\") pod \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " Jul 22 03:05:39.624689 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624527 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-home\") pod \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " Jul 22 03:05:39.624689 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624561 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdkmq\" (UniqueName: \"kubernetes.io/projected/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-kube-api-access-jdkmq\") pod \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " Jul 22 03:05:39.624689 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624596 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tls-certs\") pod \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " Jul 22 03:05:39.624689 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624623 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-dshm\") pod \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " Jul 22 03:05:39.624689 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624638 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tmp-dir\") pod \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\" (UID: \"39199532-0c15-4a91-a7ef-e9d2d91fb7ab\") " Jul 22 03:05:39.625005 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624834 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-model-cache" (OuterVolumeSpecName: "model-cache") pod "39199532-0c15-4a91-a7ef-e9d2d91fb7ab" (UID: "39199532-0c15-4a91-a7ef-e9d2d91fb7ab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:05:39.625005 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624841 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-home" (OuterVolumeSpecName: "home") pod "39199532-0c15-4a91-a7ef-e9d2d91fb7ab" (UID: "39199532-0c15-4a91-a7ef-e9d2d91fb7ab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:05:39.625005 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624960 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-model-cache\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:05:39.625005 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.624978 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-home\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:05:39.625205 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.625069 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39199532-0c15-4a91-a7ef-e9d2d91fb7ab" (UID: "39199532-0c15-4a91-a7ef-e9d2d91fb7ab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:05:39.627040 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.627015 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39199532-0c15-4a91-a7ef-e9d2d91fb7ab" (UID: "39199532-0c15-4a91-a7ef-e9d2d91fb7ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:05:39.627040 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.627033 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-dshm" (OuterVolumeSpecName: "dshm") pod "39199532-0c15-4a91-a7ef-e9d2d91fb7ab" (UID: "39199532-0c15-4a91-a7ef-e9d2d91fb7ab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:05:39.627175 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.627039 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-kube-api-access-jdkmq" (OuterVolumeSpecName: "kube-api-access-jdkmq") pod "39199532-0c15-4a91-a7ef-e9d2d91fb7ab" (UID: "39199532-0c15-4a91-a7ef-e9d2d91fb7ab"). InnerVolumeSpecName "kube-api-access-jdkmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:05:39.726365 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.726329 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdkmq\" (UniqueName: \"kubernetes.io/projected/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-kube-api-access-jdkmq\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:05:39.726365 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.726358 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tls-certs\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:05:39.726365 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.726366 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-dshm\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:05:39.726365 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:39.726374 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39199532-0c15-4a91-a7ef-e9d2d91fb7ab-tmp-dir\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:05:40.186566 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.186483 2561 generic.go:356] "Generic (PLEG): container finished" podID="39199532-0c15-4a91-a7ef-e9d2d91fb7ab" containerID="1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093" exitCode=0 Jul 22 03:05:40.186566 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.186524 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" Jul 22 03:05:40.186566 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.186537 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" event={"ID":"39199532-0c15-4a91-a7ef-e9d2d91fb7ab","Type":"ContainerDied","Data":"1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093"} Jul 22 03:05:40.186818 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.186576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx" event={"ID":"39199532-0c15-4a91-a7ef-e9d2d91fb7ab","Type":"ContainerDied","Data":"9a9febc28c0b8cd37e1094c314547535655738542b43c1cfb01d7bde5d9b102e"} Jul 22 03:05:40.186818 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.186594 2561 scope.go:117] "RemoveContainer" containerID="1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093" Jul 22 03:05:40.195144 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.195127 2561 scope.go:117] "RemoveContainer" containerID="1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093" Jul 22 03:05:40.195396 ip-10-0-134-165 kubenswrapper[2561]: E0722 03:05:40.195380 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093\": container with ID starting with 1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093 not found: ID does not exist" containerID="1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093" Jul 22 03:05:40.195443 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.195406 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093"} err="failed to get container status \"1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093\": rpc error: code = NotFound desc = could not find container \"1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093\": container with ID starting with 1266b3ce48d8788e943f860b34fd3a3f47da4d7b5c5f952541c27427d081f093 not found: ID does not exist" Jul 22 03:05:40.208636 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.208602 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx"] Jul 22 03:05:40.214144 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:40.214118 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/sub-test-model-cf82b3-kserve-78569ddc-5bwdx"] Jul 22 03:05:41.491072 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:41.491038 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:05:41.870960 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:41.870928 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39199532-0c15-4a91-a7ef-e9d2d91fb7ab" path="/var/lib/kubelet/pods/39199532-0c15-4a91-a7ef-e9d2d91fb7ab/volumes" Jul 22 03:05:41.978070 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:41.978038 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 03:05:41.983462 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:05:41.983428 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 03:06:18.676277 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.676242 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf"] Jul 22 03:06:18.676753 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.676699 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39199532-0c15-4a91-a7ef-e9d2d91fb7ab" containerName="main" Jul 22 03:06:18.676753 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.676719 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="39199532-0c15-4a91-a7ef-e9d2d91fb7ab" containerName="main" Jul 22 03:06:18.676902 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.676830 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="39199532-0c15-4a91-a7ef-e9d2d91fb7ab" containerName="main" Jul 22 03:06:18.679925 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.679905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.683504 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.683483 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"default-dockercfg-xzbgs\"" Jul 22 03:06:18.683621 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.683504 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"openshift-service-ca.crt\"" Jul 22 03:06:18.683686 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.683646 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"rate-test-model-cf82b3-kserve-self-signed-certs\"" Jul 22 03:06:18.683745 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.683694 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf82b3\"/\"kube-root-ca.crt\"" Jul 22 03:06:18.691336 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.691310 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf"] Jul 22 03:06:18.768496 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.768456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4mq2\" (UniqueName: \"kubernetes.io/projected/db31d63d-b761-48ce-a87b-a2712f03d659-kube-api-access-d4mq2\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.768496 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.768496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db31d63d-b761-48ce-a87b-a2712f03d659-tls-certs\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.768743 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.768527 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-model-cache\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.768743 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.768611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-dshm\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.768743 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.768685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-home\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.768743 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.768702 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-tmp-dir\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870043 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870000 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-model-cache\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870043 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-dshm\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870302 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-home\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870302 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870180 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-tmp-dir\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870302 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4mq2\" (UniqueName: \"kubernetes.io/projected/db31d63d-b761-48ce-a87b-a2712f03d659-kube-api-access-d4mq2\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870302 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870288 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db31d63d-b761-48ce-a87b-a2712f03d659-tls-certs\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870529 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870507 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-model-cache\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870588 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870537 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-tmp-dir\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.870642 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.870609 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-home\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.872540 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.872519 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-dshm\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.872962 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.872946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db31d63d-b761-48ce-a87b-a2712f03d659-tls-certs\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.878699 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.878675 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4mq2\" (UniqueName: \"kubernetes.io/projected/db31d63d-b761-48ce-a87b-a2712f03d659-kube-api-access-d4mq2\") pod \"rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:18.989744 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:18.989648 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:19.125805 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:19.125780 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf"] Jul 22 03:06:19.128139 ip-10-0-134-165 kubenswrapper[2561]: W0722 03:06:19.128106 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb31d63d_b761_48ce_a87b_a2712f03d659.slice/crio-18a6721243bca465cda0f1a34358e8b708ff273817478b5f94add909532fc1fe WatchSource:0}: Error finding container 18a6721243bca465cda0f1a34358e8b708ff273817478b5f94add909532fc1fe: Status 404 returned error can't find the container with id 18a6721243bca465cda0f1a34358e8b708ff273817478b5f94add909532fc1fe Jul 22 03:06:19.362359 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:19.362325 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" event={"ID":"db31d63d-b761-48ce-a87b-a2712f03d659","Type":"ContainerStarted","Data":"30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1"} Jul 22 03:06:19.362359 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:19.362364 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" event={"ID":"db31d63d-b761-48ce-a87b-a2712f03d659","Type":"ContainerStarted","Data":"18a6721243bca465cda0f1a34358e8b708ff273817478b5f94add909532fc1fe"} Jul 22 03:06:19.380553 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:19.380507 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" podStartSLOduration=1.380489504 podStartE2EDuration="1.380489504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:06:19.378473545 +0000 UTC m=+2738.014210349" watchObservedRunningTime="2026-07-22 03:06:19.380489504 +0000 UTC m=+2738.016226337" Jul 22 03:06:22.487613 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:22.487577 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:06:28.990294 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:28.990254 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:28.990693 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:28.990314 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:29.002899 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:29.002870 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:29.417846 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:29.417820 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:06:36.680584 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:36.680540 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:06:51.187802 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:51.187750 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:06:59.560583 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:06:59.560550 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf"] Jul 22 03:07:06.522284 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:06.522194 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:07:14.581433 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.581384 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" podUID="db31d63d-b761-48ce-a87b-a2712f03d659" containerName="main" containerID="cri-o://30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1" gracePeriod=45 Jul 22 03:07:14.830142 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.830117 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:07:14.878156 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878091 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-home\") pod \"db31d63d-b761-48ce-a87b-a2712f03d659\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " Jul 22 03:07:14.878156 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878133 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-model-cache\") pod \"db31d63d-b761-48ce-a87b-a2712f03d659\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " Jul 22 03:07:14.878363 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878211 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db31d63d-b761-48ce-a87b-a2712f03d659-tls-certs\") pod \"db31d63d-b761-48ce-a87b-a2712f03d659\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " Jul 22 03:07:14.878363 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878231 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-dshm\") pod \"db31d63d-b761-48ce-a87b-a2712f03d659\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " Jul 22 03:07:14.878363 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878269 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4mq2\" (UniqueName: \"kubernetes.io/projected/db31d63d-b761-48ce-a87b-a2712f03d659-kube-api-access-d4mq2\") pod \"db31d63d-b761-48ce-a87b-a2712f03d659\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " Jul 22 03:07:14.878363 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878292 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-tmp-dir\") pod \"db31d63d-b761-48ce-a87b-a2712f03d659\" (UID: \"db31d63d-b761-48ce-a87b-a2712f03d659\") " Jul 22 03:07:14.878572 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878425 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-home" (OuterVolumeSpecName: "home") pod "db31d63d-b761-48ce-a87b-a2712f03d659" (UID: "db31d63d-b761-48ce-a87b-a2712f03d659"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:14.878572 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878468 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-model-cache" (OuterVolumeSpecName: "model-cache") pod "db31d63d-b761-48ce-a87b-a2712f03d659" (UID: "db31d63d-b761-48ce-a87b-a2712f03d659"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:14.878684 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878595 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-home\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:14.878684 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878609 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-model-cache\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:14.878684 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.878637 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db31d63d-b761-48ce-a87b-a2712f03d659" (UID: "db31d63d-b761-48ce-a87b-a2712f03d659"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:14.880558 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.880531 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db31d63d-b761-48ce-a87b-a2712f03d659-kube-api-access-d4mq2" (OuterVolumeSpecName: "kube-api-access-d4mq2") pod "db31d63d-b761-48ce-a87b-a2712f03d659" (UID: "db31d63d-b761-48ce-a87b-a2712f03d659"). InnerVolumeSpecName "kube-api-access-d4mq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:14.880657 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.880583 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-dshm" (OuterVolumeSpecName: "dshm") pod "db31d63d-b761-48ce-a87b-a2712f03d659" (UID: "db31d63d-b761-48ce-a87b-a2712f03d659"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:07:14.880657 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.880596 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db31d63d-b761-48ce-a87b-a2712f03d659-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db31d63d-b761-48ce-a87b-a2712f03d659" (UID: "db31d63d-b761-48ce-a87b-a2712f03d659"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:07:14.979222 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.979186 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db31d63d-b761-48ce-a87b-a2712f03d659-tls-certs\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:14.979222 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.979217 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-dshm\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:14.979222 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.979226 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4mq2\" (UniqueName: \"kubernetes.io/projected/db31d63d-b761-48ce-a87b-a2712f03d659-kube-api-access-d4mq2\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:14.979434 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:14.979235 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db31d63d-b761-48ce-a87b-a2712f03d659-tmp-dir\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:15.624160 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.624124 2561 generic.go:356] "Generic (PLEG): container finished" podID="db31d63d-b761-48ce-a87b-a2712f03d659" containerID="30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1" exitCode=0 Jul 22 03:07:15.624623 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.624170 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" Jul 22 03:07:15.624623 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.624208 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" event={"ID":"db31d63d-b761-48ce-a87b-a2712f03d659","Type":"ContainerDied","Data":"30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1"} Jul 22 03:07:15.624623 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.624263 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf" event={"ID":"db31d63d-b761-48ce-a87b-a2712f03d659","Type":"ContainerDied","Data":"18a6721243bca465cda0f1a34358e8b708ff273817478b5f94add909532fc1fe"} Jul 22 03:07:15.624623 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.624288 2561 scope.go:117] "RemoveContainer" containerID="30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1" Jul 22 03:07:15.633431 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.633414 2561 scope.go:117] "RemoveContainer" containerID="30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1" Jul 22 03:07:15.633699 ip-10-0-134-165 kubenswrapper[2561]: E0722 03:07:15.633681 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1\": container with ID starting with 30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1 not found: ID does not exist" containerID="30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1" Jul 22 03:07:15.633748 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.633708 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1"} err="failed to get container status \"30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1\": rpc error: code = NotFound desc = could not find container \"30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1\": container with ID starting with 30d652e422f7950e47ae055115bc4faffe42e5dfb036e00fad32ebf2ccd686f1 not found: ID does not exist" Jul 22 03:07:15.645440 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.645416 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf"] Jul 22 03:07:15.650287 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.650265 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cf82b3/rate-test-model-cf82b3-kserve-6855bc7f79-g6kzf"] Jul 22 03:07:15.655318 ip-10-0-134-165 kubenswrapper[2561]: E0722 03:07:15.655296 2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb31d63d_b761_48ce_a87b_a2712f03d659.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb31d63d_b761_48ce_a87b_a2712f03d659.slice/crio-18a6721243bca465cda0f1a34358e8b708ff273817478b5f94add909532fc1fe\": RecentStats: unable to find data in memory cache]" Jul 22 03:07:15.872066 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.872029 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db31d63d-b761-48ce-a87b-a2712f03d659" path="/var/lib/kubelet/pods/db31d63d-b761-48ce-a87b-a2712f03d659/volumes" Jul 22 03:07:15.906722 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.906654 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140712170338320"] Jul 22 03:07:15.907020 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.907007 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db31d63d-b761-48ce-a87b-a2712f03d659" containerName="main" Jul 22 03:07:15.907065 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.907022 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="db31d63d-b761-48ce-a87b-a2712f03d659" containerName="main" Jul 22 03:07:15.907098 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.907078 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="db31d63d-b761-48ce-a87b-a2712f03d659" containerName="main" Jul 22 03:07:15.911376 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.911360 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:15.913622 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.913601 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-88sqm\"" Jul 22 03:07:15.916413 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.916392 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140712170338320"] Jul 22 03:07:15.987563 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:15.987531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9p29\" (UniqueName: \"kubernetes.io/projected/565e5c73-7c70-4baf-84ec-e2b34b276cde-kube-api-access-z9p29\") pod \"test-curl-7730-140712170338320\" (UID: \"565e5c73-7c70-4baf-84ec-e2b34b276cde\") " pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:16.088395 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:16.088347 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9p29\" (UniqueName: \"kubernetes.io/projected/565e5c73-7c70-4baf-84ec-e2b34b276cde-kube-api-access-z9p29\") pod \"test-curl-7730-140712170338320\" (UID: \"565e5c73-7c70-4baf-84ec-e2b34b276cde\") " pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:16.097408 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:16.097385 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9p29\" (UniqueName: \"kubernetes.io/projected/565e5c73-7c70-4baf-84ec-e2b34b276cde-kube-api-access-z9p29\") pod \"test-curl-7730-140712170338320\" (UID: \"565e5c73-7c70-4baf-84ec-e2b34b276cde\") " pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:16.221953 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:16.221871 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:16.357462 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:16.357429 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140712170338320"] Jul 22 03:07:16.359966 ip-10-0-134-165 kubenswrapper[2561]: W0722 03:07:16.359941 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod565e5c73_7c70_4baf_84ec_e2b34b276cde.slice/crio-6186cea8c5f2c920b1f52a3fd44167e8531292b972f88b86b1b87d971ba5a7b0 WatchSource:0}: Error finding container 6186cea8c5f2c920b1f52a3fd44167e8531292b972f88b86b1b87d971ba5a7b0: Status 404 returned error can't find the container with id 6186cea8c5f2c920b1f52a3fd44167e8531292b972f88b86b1b87d971ba5a7b0 Jul 22 03:07:16.361362 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:16.361342 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:07:16.631467 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:16.631428 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140712170338320" event={"ID":"565e5c73-7c70-4baf-84ec-e2b34b276cde","Type":"ContainerStarted","Data":"6186cea8c5f2c920b1f52a3fd44167e8531292b972f88b86b1b87d971ba5a7b0"} Jul 22 03:07:17.637515 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:17.637487 2561 generic.go:356] "Generic (PLEG): container finished" podID="565e5c73-7c70-4baf-84ec-e2b34b276cde" containerID="d401af90399e5409a0eefed3d43f206d5fc3868dcdb47bdd3d2313575ceddbe2" exitCode=0 Jul 22 03:07:17.637927 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:17.637541 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140712170338320" event={"ID":"565e5c73-7c70-4baf-84ec-e2b34b276cde","Type":"ContainerDied","Data":"d401af90399e5409a0eefed3d43f206d5fc3868dcdb47bdd3d2313575ceddbe2"} Jul 22 03:07:18.769423 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.769397 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:18.805262 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.805241 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140712170338320_565e5c73-7c70-4baf-84ec-e2b34b276cde/test-curl-7730-140712170338320/0.log" Jul 22 03:07:18.812551 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.812530 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9p29\" (UniqueName: \"kubernetes.io/projected/565e5c73-7c70-4baf-84ec-e2b34b276cde-kube-api-access-z9p29\") pod \"565e5c73-7c70-4baf-84ec-e2b34b276cde\" (UID: \"565e5c73-7c70-4baf-84ec-e2b34b276cde\") " Jul 22 03:07:18.814788 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.814746 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565e5c73-7c70-4baf-84ec-e2b34b276cde-kube-api-access-z9p29" (OuterVolumeSpecName: "kube-api-access-z9p29") pod "565e5c73-7c70-4baf-84ec-e2b34b276cde" (UID: "565e5c73-7c70-4baf-84ec-e2b34b276cde"). InnerVolumeSpecName "kube-api-access-z9p29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:18.827864 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.827840 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140712170338320"] Jul 22 03:07:18.830053 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.830032 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140712170338320"] Jul 22 03:07:18.914034 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:18.913979 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9p29\" (UniqueName: \"kubernetes.io/projected/565e5c73-7c70-4baf-84ec-e2b34b276cde-kube-api-access-z9p29\") on node \"ip-10-0-134-165.ec2.internal\" DevicePath \"\"" Jul 22 03:07:19.648376 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:19.648351 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:19.648376 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:19.648369 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6186cea8c5f2c920b1f52a3fd44167e8531292b972f88b86b1b87d971ba5a7b0" Jul 22 03:07:19.658169 ip-10-0-134-165 kubenswrapper[2561]: E0722 03:07:19.658136 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7730-140712170338320\" is forbidden: User \"system:node:ip-10-0-134-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-165.ec2.internal' and this object" podUID="565e5c73-7c70-4baf-84ec-e2b34b276cde" pod="opendatahub/test-curl-7730-140712170338320" Jul 22 03:07:19.870318 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:19.870283 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="565e5c73-7c70-4baf-84ec-e2b34b276cde" path="/var/lib/kubelet/pods/565e5c73-7c70-4baf-84ec-e2b34b276cde/volumes" Jul 22 03:07:32.800169 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:32.800138 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:07:54.291655 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:07:54.291615 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggsmp"] Jul 22 03:08:03.468912 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:03.468875 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc4cf4958-zrts4_25bc005b-d35e-4827-b1b6-1c2c65395b59/authorino/0.log" Jul 22 03:08:09.636199 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:09.636170 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fcd77b786-z9862_579a6e76-66ea-4dda-941e-66041b73a322/manager/0.log" Jul 22 03:08:09.642522 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:09.642494 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fcd77b786-z9862_579a6e76-66ea-4dda-941e-66041b73a322/copy-manifests/0.log" Jul 22 03:08:11.783005 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:11.782963 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc4cf4958-zrts4_25bc005b-d35e-4827-b1b6-1c2c65395b59/authorino/0.log" Jul 22 03:08:12.022220 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:12.022187 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s2qcw_4d183f4e-b01e-4100-a679-aa3631ffe6d5/manager/0.log" Jul 22 03:08:12.134111 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:12.134082 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2l9kp_1ab77e94-aab4-4d72-8015-4ca43445c14c/kuadrant-console-plugin/0.log" Jul 22 03:08:12.499892 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:12.499801 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ggsmp_eeba1e6f-97b2-404e-b207-ddbfb082391c/limitador/0.log" Jul 22 03:08:13.345653 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:13.345602 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79676bf5cc-lj2z6_4a996b46-59b1-4f27-b843-4023bd3892b2/kube-auth-proxy/0.log" Jul 22 03:08:13.831175 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:13.831147 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c448899f8-d7r6m_13da7e77-aeeb-4c25-a7ed-512fc1407a71/router/0.log" Jul 22 03:08:14.174520 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:14.174433 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jzk7x_c60f9ba0-84a4-4bae-9aeb-ebbdf4548c4e/main/0.log" Jul 22 03:08:14.506593 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:14.506514 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd9zvt_21622bfe-9a67-40d5-8f26-5e0bebc6cb87/main/0.log" Jul 22 03:08:14.727802 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:14.727760 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-npzjz_47dbe9de-d859-47eb-9caf-30d48a2483f8/main/0.log" Jul 22 03:08:22.900413 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:22.900382 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k6hvt_e149137e-7472-41b4-85a0-99c095e9db20/global-pull-secret-syncer/0.log" Jul 22 03:08:23.097815 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:23.097789 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wmhdc_72cca779-c7b3-4615-9133-3aa76401f137/konnectivity-agent/0.log" Jul 22 03:08:23.123605 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:23.123567 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-165.ec2.internal_2ed18d9f9ede21e451f5e83ef12c968d/haproxy/0.log" Jul 22 03:08:27.796350 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:27.796312 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc4cf4958-zrts4_25bc005b-d35e-4827-b1b6-1c2c65395b59/authorino/0.log" Jul 22 03:08:27.874693 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:27.874666 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s2qcw_4d183f4e-b01e-4100-a679-aa3631ffe6d5/manager/0.log" Jul 22 03:08:27.903179 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:27.903155 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2l9kp_1ab77e94-aab4-4d72-8015-4ca43445c14c/kuadrant-console-plugin/0.log" Jul 22 03:08:28.223442 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:28.223413 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ggsmp_eeba1e6f-97b2-404e-b207-ddbfb082391c/limitador/0.log" Jul 22 03:08:30.249959 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.249881 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-pldkp_c23179af-03e0-4ab8-b9c8-e95baa2529d4/cluster-monitoring-operator/0.log" Jul 22 03:08:30.421648 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.421622 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dzx8b_d9d1298d-8de9-483e-9ca1-52d3a12d8711/node-exporter/0.log" Jul 22 03:08:30.441089 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.441068 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dzx8b_d9d1298d-8de9-483e-9ca1-52d3a12d8711/kube-rbac-proxy/0.log" Jul 22 03:08:30.460560 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.460540 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dzx8b_d9d1298d-8de9-483e-9ca1-52d3a12d8711/init-textfile/0.log" Jul 22 03:08:30.770521 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.770494 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/prometheus/0.log" Jul 22 03:08:30.793410 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.793356 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/config-reloader/0.log" Jul 22 03:08:30.823002 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.822968 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/thanos-sidecar/0.log" Jul 22 03:08:30.849890 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.849822 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/kube-rbac-proxy-web/0.log" Jul 22 03:08:30.884134 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.884090 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/kube-rbac-proxy/0.log" Jul 22 03:08:30.914406 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.914383 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/kube-rbac-proxy-thanos/0.log" Jul 22 03:08:30.941453 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:30.941433 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a8a64a0d-ebba-4dbd-a625-6e62579b1bb2/init-config-reloader/0.log" Jul 22 03:08:32.385166 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:32.385135 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-v5zg6_df32fe08-cfc7-4388-b46e-50d793f5522f/networking-console-plugin/0.log" Jul 22 03:08:33.023899 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.023842 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/1.log" Jul 22 03:08:33.029479 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.029452 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2"] Jul 22 03:08:33.029799 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.029756 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pt7lk_17d6253b-91b2-43e4-b9fc-3744f8ce11ca/console-operator/2.log" Jul 22 03:08:33.029896 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.029881 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565e5c73-7c70-4baf-84ec-e2b34b276cde" containerName="test-curl-7730-140712170338320" Jul 22 03:08:33.029939 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.029899 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="565e5c73-7c70-4baf-84ec-e2b34b276cde" containerName="test-curl-7730-140712170338320" Jul 22 03:08:33.029970 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.029959 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="565e5c73-7c70-4baf-84ec-e2b34b276cde" containerName="test-curl-7730-140712170338320" Jul 22 03:08:33.033100 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.033082 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.036045 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.036021 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h4tn6\"/\"openshift-service-ca.crt\"" Jul 22 03:08:33.036175 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.036030 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h4tn6\"/\"default-dockercfg-cr7fc\"" Jul 22 03:08:33.036700 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.036681 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h4tn6\"/\"kube-root-ca.crt\"" Jul 22 03:08:33.043418 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.043396 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2"] Jul 22 03:08:33.149267 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.149183 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-proc\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.149267 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.149237 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-lib-modules\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.149486 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.149312 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-podres\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.149486 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.149353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvm42\" (UniqueName: \"kubernetes.io/projected/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-kube-api-access-vvm42\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.149486 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.149431 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-sys\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250022 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.249970 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-lib-modules\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250022 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-podres\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250063 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvm42\" (UniqueName: \"kubernetes.io/projected/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-kube-api-access-vvm42\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250095 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-sys\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-proc\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250189 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-lib-modules\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250200 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-sys\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250201 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-podres\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.250295 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.250197 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-proc\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.258636 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.258617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvm42\" (UniqueName: \"kubernetes.io/projected/ddc5a2a6-5a65-407c-b8e9-cd0e92952a28-kube-api-access-vvm42\") pod \"perf-node-gather-daemonset-wvfd2\" (UID: \"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28\") " pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.343453 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.343425 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:33.480580 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.480554 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2"] Jul 22 03:08:33.482578 ip-10-0-134-165 kubenswrapper[2561]: W0722 03:08:33.482550 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podddc5a2a6_5a65_407c_b8e9_cd0e92952a28.slice/crio-0a249d1ae35844342d8fcb5e330e5bdbc8e34a663ea0abd638a91b76541eb673 WatchSource:0}: Error finding container 0a249d1ae35844342d8fcb5e330e5bdbc8e34a663ea0abd638a91b76541eb673: Status 404 returned error can't find the container with id 0a249d1ae35844342d8fcb5e330e5bdbc8e34a663ea0abd638a91b76541eb673 Jul 22 03:08:33.987712 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.987675 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" event={"ID":"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28","Type":"ContainerStarted","Data":"c59102568157e8d03e1dff142100c4ae7f0a2ad5e345fb2c0f994a84c77ad667"} Jul 22 03:08:33.987922 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.987737 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" event={"ID":"ddc5a2a6-5a65-407c-b8e9-cd0e92952a28","Type":"ContainerStarted","Data":"0a249d1ae35844342d8fcb5e330e5bdbc8e34a663ea0abd638a91b76541eb673"} Jul 22 03:08:33.987922 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:33.987802 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:34.007585 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:34.007538 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" podStartSLOduration=1.007524642 podStartE2EDuration="1.007524642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:34.006215857 +0000 UTC m=+2872.641952656" watchObservedRunningTime="2026-07-22 03:08:34.007524642 +0000 UTC m=+2872.643261499" Jul 22 03:08:34.208827 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:34.208798 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-tx6g8_7a467496-665d-4f59-9b58-8d384fba5652/volume-data-source-validator/0.log" Jul 22 03:08:35.219954 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:35.219920 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-chnzz_69f61c23-1cb1-4e11-8166-6ac5a80d71a5/dns/0.log" Jul 22 03:08:35.241878 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:35.241853 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-chnzz_69f61c23-1cb1-4e11-8166-6ac5a80d71a5/kube-rbac-proxy/0.log" Jul 22 03:08:35.360312 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:35.360288 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-564zk_66dde4a9-a236-4bda-a7e2-2da7d71d668d/dns-node-resolver/0.log" Jul 22 03:08:36.007736 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:36.007708 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8z26_f29107fe-f689-4b7d-9b0b-39375612a435/node-ca/0.log" Jul 22 03:08:37.171443 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:37.171411 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79676bf5cc-lj2z6_4a996b46-59b1-4f27-b843-4023bd3892b2/kube-auth-proxy/0.log" Jul 22 03:08:37.348561 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:37.348534 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c448899f8-d7r6m_13da7e77-aeeb-4c25-a7ed-512fc1407a71/router/0.log" Jul 22 03:08:38.065228 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:38.065196 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdf5m_defc9a21-3c7c-49c8-9ced-49644ae3954b/serve-healthcheck-canary/0.log" Jul 22 03:08:38.537732 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:38.537644 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-st5bj_f7c31c9a-a2fc-4f93-a034-805ba7708a21/insights-operator/0.log" Jul 22 03:08:38.538536 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:38.538520 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-st5bj_f7c31c9a-a2fc-4f93-a034-805ba7708a21/insights-operator/1.log" Jul 22 03:08:38.689839 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:38.689807 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp268_bd608150-abbf-4063-8857-09d0b082205f/kube-rbac-proxy/0.log" Jul 22 03:08:38.711380 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:38.711358 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp268_bd608150-abbf-4063-8857-09d0b082205f/exporter/0.log" Jul 22 03:08:38.731647 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:38.731619 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xp268_bd608150-abbf-4063-8857-09d0b082205f/extractor/0.log" Jul 22 03:08:40.002549 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:40.002515 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h4tn6/perf-node-gather-daemonset-wvfd2" Jul 22 03:08:41.429030 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:41.429004 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fcd77b786-z9862_579a6e76-66ea-4dda-941e-66041b73a322/manager/0.log" Jul 22 03:08:41.467725 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:41.467695 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fcd77b786-z9862_579a6e76-66ea-4dda-941e-66041b73a322/copy-manifests/0.log" Jul 22 03:08:43.009148 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:43.009067 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6b78567f4c-vt2cq_6e72b07b-6a13-4dff-9cfc-264d3c83819b/manager/0.log" Jul 22 03:08:43.385726 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:43.385699 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79676bf5cc-lj2z6_4a996b46-59b1-4f27-b843-4023bd3892b2/kube-auth-proxy/0.log" Jul 22 03:08:43.509841 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:43.509811 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c448899f8-d7r6m_13da7e77-aeeb-4c25-a7ed-512fc1407a71/router/0.log" Jul 22 03:08:44.933553 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:44.933531 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79676bf5cc-lj2z6_4a996b46-59b1-4f27-b843-4023bd3892b2/kube-auth-proxy/0.log" Jul 22 03:08:45.058043 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:45.057997 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c448899f8-d7r6m_13da7e77-aeeb-4c25-a7ed-512fc1407a71/router/0.log" Jul 22 03:08:48.536468 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:48.536437 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-2g47p_4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3/kube-storage-version-migrator-operator/1.log" Jul 22 03:08:48.537185 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:48.537165 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-2g47p_4b6dd54b-a535-4fe1-b9e9-cd900c1fafa3/kube-storage-version-migrator-operator/0.log" Jul 22 03:08:49.582288 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.582254 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/kube-multus-additional-cni-plugins/0.log" Jul 22 03:08:49.606590 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.606560 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/egress-router-binary-copy/0.log" Jul 22 03:08:49.631109 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.631080 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/cni-plugins/0.log" Jul 22 03:08:49.653646 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.653620 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/bond-cni-plugin/0.log" Jul 22 03:08:49.677424 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.677393 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/routeoverride-cni/0.log" Jul 22 03:08:49.699654 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.699627 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/whereabouts-cni-bincopy/0.log" Jul 22 03:08:49.720842 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:49.720812 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgnn4_61382fc5-f560-4435-a9f0-d0dc961dbe30/whereabouts-cni/0.log" Jul 22 03:08:50.183387 ip-10-0-134-165 kubenswrapper[2561]: I0722 03:08:50.183362 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zl86t_e80114d7-e030-4d8f-98bb-f99f3d6d51b8/kube-multus/0.log"