Jul 09 18:58:18.028148 ip-10-0-133-17 systemd[1]: Starting Kubernetes Kubelet... Jul 09 18:58:18.473020 ip-10-0-133-17 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:58:18.473020 ip-10-0-133-17 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 18:58:18.473020 ip-10-0-133-17 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:58:18.473020 ip-10-0-133-17 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 18:58:18.473020 ip-10-0-133-17 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:58:18.475329 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.475080 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 18:58:18.480005 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.479990 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:58:18.480005 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480006 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480010 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480014 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480017 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480020 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480023 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480026 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480030 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480033 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480036 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480039 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480041 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480044 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480046 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480049 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480052 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480054 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480057 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480059 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480063 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480065 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:58:18.480071 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480068 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480078 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480081 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480084 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480087 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480089 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480092 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480096 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480100 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480102 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480105 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480107 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480110 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480115 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480119 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480122 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480125 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480127 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480130 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:58:18.480564 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480133 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480135 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480138 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480140 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480143 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480145 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480148 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480150 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480153 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480155 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480158 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480160 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480164 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480168 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480171 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480175 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480177 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480180 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480183 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:58:18.481036 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480186 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480189 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480192 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480194 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480197 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480199 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480202 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480205 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480207 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480210 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480212 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480216 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480218 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480221 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480223 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480226 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480229 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480231 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480234 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:58:18.481492 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480237 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480239 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480242 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480244 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480247 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480250 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480252 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480255 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480258 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480262 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480264 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480267 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480270 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480273 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480276 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480278 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480281 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480284 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480286 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.480289 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:58:18.481958 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481006 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481013 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481016 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481019 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481022 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481024 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481027 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481030 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481033 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481036 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481039 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481042 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481045 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481047 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481050 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481053 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481055 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481058 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481060 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481063 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:58:18.482428 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481066 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481069 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481072 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481074 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481077 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481080 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481084 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481088 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481091 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481094 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481096 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481099 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481102 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481105 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481108 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481112 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481115 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481118 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481121 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:58:18.482936 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481123 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481126 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481129 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481131 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481134 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481137 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481139 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481141 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481145 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481148 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481150 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481153 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481157 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481160 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481162 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481165 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481168 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481170 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481173 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481175 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:58:18.483407 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481178 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481180 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481182 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481185 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481188 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481190 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481193 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481198 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481201 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481204 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481207 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481210 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481213 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481216 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481219 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481222 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481225 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481228 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481230 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:58:18.483912 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481233 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481235 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481238 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481240 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481243 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481246 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481248 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481251 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481254 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481256 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481258 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481261 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481264 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481266 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481269 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481271 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481274 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481276 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481278 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481281 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:58:18.484374 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481283 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481648 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481652 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481656 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481659 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481662 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481665 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481667 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481670 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481673 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481675 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481678 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481680 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481683 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481686 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481688 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481691 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481694 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481697 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:58:18.484867 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481700 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481702 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481705 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481708 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481710 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481713 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481716 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481718 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481721 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481723 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481725 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481728 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481731 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481734 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481736 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481739 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481741 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481744 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481746 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481749 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:58:18.485325 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481752 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481754 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481757 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481759 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481762 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481766 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481770 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481773 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481776 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481778 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481781 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481784 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481787 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481789 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481792 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481794 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481796 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481799 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481803 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481807 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:58:18.485842 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481809 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481812 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481829 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481832 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481835 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481840 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481843 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481846 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481849 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481852 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481858 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481861 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481864 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481867 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481870 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481873 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481875 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481878 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481880 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:58:18.486327 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481882 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481885 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481888 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481892 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481895 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481897 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481900 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481902 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481905 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481907 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481910 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481913 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481915 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481918 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481920 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481923 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481925 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481928 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481930 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481933 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:58:18.486834 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481935 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.481938 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482010 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482018 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482027 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482031 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482036 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482039 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482043 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482048 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482051 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482054 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482057 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482061 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482064 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482068 2561 flags.go:64] FLAG: --cgroup-root="" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482071 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482074 2561 flags.go:64] FLAG: --client-ca-file="" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482077 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482080 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482083 2561 flags.go:64] FLAG: --cluster-domain="" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482086 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482089 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482092 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 18:58:18.487335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482096 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482099 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482102 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482105 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482109 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482112 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482115 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482117 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482120 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482123 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482127 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482130 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482133 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482136 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482140 2561 flags.go:64] FLAG: --enable-server="true" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482143 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482148 2561 flags.go:64] FLAG: --event-burst="100" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482151 2561 flags.go:64] FLAG: --event-qps="50" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482154 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482157 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482160 2561 flags.go:64] FLAG: --eviction-hard="" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482164 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482167 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482171 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482174 2561 flags.go:64] FLAG: --eviction-soft="" Jul 09 18:58:18.487920 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482177 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482180 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482183 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482186 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482189 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482192 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482195 2561 flags.go:64] FLAG: --feature-gates="" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482199 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482202 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482205 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482209 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482212 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482215 2561 flags.go:64] FLAG: --help="false" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482218 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482221 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482224 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482227 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482231 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482234 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482237 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482240 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482243 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482246 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 18:58:18.488639 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482250 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482253 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482256 2561 flags.go:64] FLAG: --kube-reserved="" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482259 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482263 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482266 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482269 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482273 2561 flags.go:64] FLAG: --lock-file="" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482277 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482280 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482283 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482288 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482291 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482294 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482297 2561 flags.go:64] FLAG: --logging-format="text" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482300 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482303 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482306 2561 flags.go:64] FLAG: --manifest-url="" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482309 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482315 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482318 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482322 2561 flags.go:64] FLAG: --max-pods="110" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482325 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482328 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482331 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 18:58:18.489245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482334 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482337 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482340 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482343 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482350 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482353 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482356 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482359 2561 flags.go:64] FLAG: --pod-cidr="" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482362 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482368 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482371 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482374 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482377 2561 flags.go:64] FLAG: --port="10250" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482379 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482384 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09dcc2801f784759e" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482387 2561 flags.go:64] FLAG: --qos-reserved="" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482390 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482392 2561 flags.go:64] FLAG: --register-node="true" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482395 2561 flags.go:64] FLAG: --register-with-taints="" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482399 2561 flags.go:64] FLAG: --registry-burst="10" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482402 2561 flags.go:64] FLAG: --registry-qps="5" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482404 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482407 2561 flags.go:64] FLAG: --reserved-memory="" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482411 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482414 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 18:58:18.490163 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482417 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482419 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482422 2561 flags.go:64] FLAG: --runonce="false" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482425 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482429 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482432 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482434 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482437 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482441 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482444 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482449 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482452 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482455 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482458 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482461 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482464 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482467 2561 flags.go:64] FLAG: --system-cgroups="" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482470 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482475 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482478 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482481 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482486 2561 flags.go:64] FLAG: --tls-min-version="" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482488 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482492 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482494 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 18:58:18.490776 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482497 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482500 2561 flags.go:64] FLAG: --v="2" Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482504 2561 flags.go:64] FLAG: --version="false" Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482508 2561 flags.go:64] FLAG: --vmodule="" Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482511 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.482515 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482627 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482631 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482634 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482638 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482641 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482643 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482646 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482649 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482651 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482654 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482656 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482660 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482663 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482666 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482668 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:58:18.491854 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482671 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482674 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482676 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482679 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482682 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482685 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482687 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482690 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482693 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482697 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482700 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482704 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482707 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482710 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482712 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482715 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482718 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482721 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482723 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:58:18.492680 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482726 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482728 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482731 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482733 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482736 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482738 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482740 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482743 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482745 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482749 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482751 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482754 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482757 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482759 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482762 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482765 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482767 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482770 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482773 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482776 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:58:18.493200 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482778 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482781 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482783 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482786 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482788 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482791 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482793 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482795 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482798 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482800 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482803 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482805 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482808 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482810 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482827 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482831 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482835 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482837 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482840 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482843 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:58:18.493883 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482846 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482850 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482853 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482855 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482858 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482861 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482864 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482866 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482869 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482871 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482875 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482878 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482880 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482883 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482885 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482888 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482890 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482894 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482898 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:58:18.494749 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482901 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482903 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482906 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482909 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482911 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.482914 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.483625 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.492523 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.492545 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.493492 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.493512 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493626 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493635 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493640 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493645 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:58:18.495367 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493649 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493654 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493658 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493662 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493667 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493671 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493676 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493681 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493685 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493690 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493697 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493702 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493706 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493710 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493715 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493719 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493723 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493728 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493733 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493737 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:58:18.495828 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493741 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493745 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493749 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493754 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493758 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493771 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493775 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493780 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493784 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493788 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493792 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493797 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493801 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493805 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493809 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493830 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493835 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493839 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493843 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493847 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:58:18.496380 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493853 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493857 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493861 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493866 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493870 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493874 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493878 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493882 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493886 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493890 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493894 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493898 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493902 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493906 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493914 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493919 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493924 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493931 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493937 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:58:18.497091 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493942 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493946 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493950 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493954 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493958 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493963 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493967 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493972 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493976 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493980 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493984 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493988 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493993 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.493997 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494001 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494005 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494009 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494013 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494018 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494022 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:58:18.497906 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494026 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494030 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494034 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494039 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494043 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494046 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494053 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494059 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494063 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494068 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494073 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494078 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494083 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494087 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494091 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:58:18.498687 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494095 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.494104 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494284 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494292 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494297 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494302 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494307 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494311 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494316 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494320 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494325 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494329 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494336 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494342 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:58:18.499279 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494347 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494352 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494356 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494360 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494364 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494368 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494374 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494379 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494384 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494388 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494392 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494396 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494400 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494405 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494409 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494413 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494420 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494424 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494429 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494434 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:58:18.499665 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494440 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494445 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494449 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494453 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494457 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494462 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494466 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494471 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494476 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494480 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494485 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494488 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494493 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494497 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494501 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494505 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494510 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494513 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494518 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494522 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:58:18.500180 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494526 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494530 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494534 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494538 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494542 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494547 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494552 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494557 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494561 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494566 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494570 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494574 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494578 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494582 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494586 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494590 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494594 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494598 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494602 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494606 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:58:18.500684 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494610 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494614 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494618 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494622 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494626 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494630 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494634 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494638 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494642 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494646 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494650 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494654 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494658 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494663 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494667 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494671 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494676 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494680 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494685 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494689 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:58:18.501188 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494693 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494697 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494702 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494706 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494710 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494714 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:18.494718 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.494727 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.494911 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 18:58:18.501698 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.500612 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 18:58:18.502323 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.502310 2561 server.go:1019] "Starting client certificate rotation" Jul 09 18:58:18.502436 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.502418 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:58:18.502468 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.502452 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:58:18.505132 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.505119 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:58:18.509130 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.509115 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:58:18.521217 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.521200 2561 log.go:25] "Validated CRI v1 runtime API" Jul 09 18:58:18.523828 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.523791 2561 log.go:25] "Validated CRI v1 image API" Jul 09 18:58:18.525449 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.525422 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 18:58:18.528070 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.528038 2561 fs.go:135] Filesystem UUIDs: map[0eac688f-3cdc-4dea-ae74-4e788a5fab71:/dev/nvme0n1p4 4b82267f-2e00-42ba-9f6f-ab126bf17f43:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 09 18:58:18.528070 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.528064 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 18:58:18.533444 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.533327 2561 manager.go:216] Machine: {Timestamp:2026-07-09 18:58:18.531961679 +0000 UTC m=+0.376103149 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096827 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29eaea8934f328ac5ba9e4d2582f7a SystemUUID:ec29eaea-8934-f328-ac5b-a9e4d2582f7a BootID:d5c73d0a-2acb-44a5-9a0e-a181db65aea5 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:67:fc:ea:7b:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:67:fc:ea:7b:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:61:d9:c6:9d:f9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 18:58:18.533444 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.533432 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 18:58:18.533576 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.533562 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 18:58:18.534405 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.534389 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:58:18.536151 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.536121 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 18:58:18.536297 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.536153 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 18:58:18.537036 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.537025 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 18:58:18.537072 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.537038 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 18:58:18.537072 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.537046 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:58:18.537072 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.537059 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:58:18.538225 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.538216 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 18:58:18.541769 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541758 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:58:18.541896 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541886 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 18:58:18.541933 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541917 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 09 18:58:18.541933 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541927 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 18:58:18.541988 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541937 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 18:58:18.541988 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541946 2561 kubelet.go:398] "Adding apiserver pod source" Jul 09 18:58:18.541988 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.541954 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 18:58:18.542327 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.542315 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:58:18.542363 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.542340 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:58:18.545249 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.545223 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 18:58:18.547999 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.547981 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 18:58:18.548088 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548022 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 18:58:18.548778 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548767 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 18:58:18.548843 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548784 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 18:58:18.548843 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548791 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 18:58:18.548843 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548797 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 18:58:18.548843 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548805 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 18:58:18.548843 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548811 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 18:58:18.548843 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548842 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548848 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548856 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548862 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548871 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548879 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548901 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 18:58:18.549000 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.548906 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 18:58:18.552210 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.552197 2561 server.go:1268] "Started kubelet" Jul 09 18:58:18.552335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.552283 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 18:58:18.552510 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.552287 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 18:58:18.552557 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.552532 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 18:58:18.553537 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.553517 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 18:58:18.553749 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.553715 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 18:58:18.553737 ip-10-0-133-17 systemd[1]: Started Kubernetes Kubelet. Jul 09 18:58:18.554019 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.553753 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 18:58:18.554678 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.554512 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 09 18:58:18.554678 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.554598 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 18:58:18.558618 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.557613 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-17.ec2.internal.18c0b41f343dd4af default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-17.ec2.internal,UID:ip-10-0-133-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-17.ec2.internal,},FirstTimestamp:2026-07-09 18:58:18.552169647 +0000 UTC m=+0.396311115,LastTimestamp:2026-07-09 18:58:18.552169647 +0000 UTC m=+0.396311115,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-17.ec2.internal,}" Jul 09 18:58:18.559699 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.559679 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 18:58:18.563366 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.563352 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 18:58:18.563424 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.563367 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 18:58:18.563537 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.563513 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:18.564052 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564033 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 18:58:18.564124 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564060 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 18:58:18.564202 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564183 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 09 18:58:18.564202 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564195 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 09 18:58:18.564324 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564230 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 18:58:18.564355 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564343 2561 factory.go:55] Registering systemd factory Jul 09 18:58:18.564383 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564355 2561 factory.go:223] Registration of the systemd container factory successfully Jul 09 18:58:18.564610 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564596 2561 factory.go:153] Registering CRI-O factory Jul 09 18:58:18.564683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564614 2561 factory.go:223] Registration of the crio container factory successfully Jul 09 18:58:18.564683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564667 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 18:58:18.564763 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564692 2561 factory.go:103] Registering Raw factory Jul 09 18:58:18.564763 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.564709 2561 manager.go:1196] Started watching for new ooms in manager Jul 09 18:58:18.565207 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.565192 2561 manager.go:349] Starting recovery of all containers Jul 09 18:58:18.574932 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.574907 2561 manager.go:354] Recovery completed Jul 09 18:58:18.577323 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.577294 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 18:58:18.577414 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.577325 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 18:58:18.580189 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.580174 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.581768 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.581753 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-msvnj" Jul 09 18:58:18.582447 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.582428 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.582499 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.582463 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.582499 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.582474 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.582991 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.582974 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 18:58:18.582991 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.582990 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 18:58:18.583113 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.583009 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:58:18.583113 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.583036 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.584208 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.584194 2561 policy_none.go:49] "None policy: Start" Jul 09 18:58:18.584271 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.584219 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 18:58:18.584271 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.584237 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 18:58:18.584526 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.584512 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.584573 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.584539 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.584573 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.584548 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.585380 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.585368 2561 policy_none.go:47] "Start" Jul 09 18:58:18.587609 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.587590 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-msvnj" Jul 09 18:58:18.592950 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.592881 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-17.ec2.internal.18c0b41f360bdb94 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-17.ec2.internal,UID:ip-10-0-133-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-17.ec2.internal,},FirstTimestamp:2026-07-09 18:58:18.582449044 +0000 UTC m=+0.426590515,LastTimestamp:2026-07-09 18:58:18.582449044 +0000 UTC m=+0.426590515,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-17.ec2.internal,}" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629404 2561 manager.go:337] "Starting Device Plugin manager" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.629437 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629447 2561 server.go:87] "Starting device plugin registration server" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629709 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629719 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629832 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629910 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.629917 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.630468 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 18:58:18.632076 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.630532 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:18.699798 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.699756 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 18:58:18.701247 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.701230 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 18:58:18.701298 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.701269 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 18:58:18.701298 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.701289 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 18:58:18.701383 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.701369 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 18:58:18.704282 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.704256 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:58:18.730607 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.730540 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.733710 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.733692 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.733809 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.733730 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.733809 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.733744 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.733809 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.733773 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.746036 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.746012 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.746136 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.746040 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-17.ec2.internal\": node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:18.768769 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.768741 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:18.801692 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.801656 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal"] Jul 09 18:58:18.801875 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.801742 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.802722 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.802704 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.802794 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.802736 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.802794 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.802746 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.803976 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.803963 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.804103 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.804085 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.804136 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.804122 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.805407 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.805391 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.805473 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.805418 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.805473 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.805428 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.805548 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.805483 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.805548 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.805502 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.805548 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.805512 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.806628 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.806610 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.806718 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.806635 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:18.807307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.807290 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:18.807404 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.807320 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:18.807404 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.807363 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:18.840570 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.840548 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-17.ec2.internal\" not found" node="ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.845006 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.844990 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-17.ec2.internal\" not found" node="ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.865501 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.865476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/752495a5e2997f2598ef0fe538e1c710-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal\" (UID: \"752495a5e2997f2598ef0fe538e1c710\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.865609 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.865510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/752495a5e2997f2598ef0fe538e1c710-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal\" (UID: \"752495a5e2997f2598ef0fe538e1c710\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.865609 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.865528 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2167be909be366b8f502c0ed2d1d4ffd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-17.ec2.internal\" (UID: \"2167be909be366b8f502c0ed2d1d4ffd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.869583 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.869566 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:18.966405 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.966361 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/752495a5e2997f2598ef0fe538e1c710-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal\" (UID: \"752495a5e2997f2598ef0fe538e1c710\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.966492 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.966410 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/752495a5e2997f2598ef0fe538e1c710-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal\" (UID: \"752495a5e2997f2598ef0fe538e1c710\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.966492 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.966429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2167be909be366b8f502c0ed2d1d4ffd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-17.ec2.internal\" (UID: \"2167be909be366b8f502c0ed2d1d4ffd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.966492 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.966460 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/752495a5e2997f2598ef0fe538e1c710-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal\" (UID: \"752495a5e2997f2598ef0fe538e1c710\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.966492 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.966468 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/752495a5e2997f2598ef0fe538e1c710-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal\" (UID: \"752495a5e2997f2598ef0fe538e1c710\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.966608 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:18.966461 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2167be909be366b8f502c0ed2d1d4ffd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-17.ec2.internal\" (UID: \"2167be909be366b8f502c0ed2d1d4ffd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" Jul 09 18:58:18.970469 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:18.970454 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.071370 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.071296 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.142544 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.142506 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:19.147879 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.147859 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" Jul 09 18:58:19.172050 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.171859 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.272474 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.272429 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.373001 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.372921 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.473574 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.473539 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.501681 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.501643 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 18:58:19.501858 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.501837 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 18:58:19.574315 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.574287 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.591690 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.591629 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 18:53:19 +0000 UTC" deadline="2028-03-20 01:26:38.420915666 +0000 UTC" Jul 09 18:58:19.591690 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.591683 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14862h28m18.829235193s" Jul 09 18:58:19.591690 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.591693 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:58:19.674857 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.674747 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:19.697703 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:19.697656 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752495a5e2997f2598ef0fe538e1c710.slice/crio-2ce28207d4c110458e96785f921a337fbf062e805ee3f15436edb95cc2ae424c WatchSource:0}: Error finding container 2ce28207d4c110458e96785f921a337fbf062e805ee3f15436edb95cc2ae424c: Status 404 returned error can't find the container with id 2ce28207d4c110458e96785f921a337fbf062e805ee3f15436edb95cc2ae424c Jul 09 18:58:19.698299 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:19.698274 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2167be909be366b8f502c0ed2d1d4ffd.slice/crio-a3df8d341e1f94c3462569447dd44625264e94c192608f37dcf1d512feae5afe WatchSource:0}: Error finding container a3df8d341e1f94c3462569447dd44625264e94c192608f37dcf1d512feae5afe: Status 404 returned error can't find the container with id a3df8d341e1f94c3462569447dd44625264e94c192608f37dcf1d512feae5afe Jul 09 18:58:19.702082 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.702067 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:58:19.704335 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.704292 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" event={"ID":"2167be909be366b8f502c0ed2d1d4ffd","Type":"ContainerStarted","Data":"a3df8d341e1f94c3462569447dd44625264e94c192608f37dcf1d512feae5afe"} Jul 09 18:58:19.705440 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.705418 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" event={"ID":"752495a5e2997f2598ef0fe538e1c710","Type":"ContainerStarted","Data":"2ce28207d4c110458e96785f921a337fbf062e805ee3f15436edb95cc2ae424c"} Jul 09 18:58:19.707547 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.707531 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:58:19.708916 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.708900 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:58:19.708990 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.708928 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:58:19.708990 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:19.708942 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:58:19.775285 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:19.775245 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:21.268453 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:21.268414 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:21.369015 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:21.368934 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-17.ec2.internal\" not found" Jul 09 18:58:21.409008 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.408986 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:58:21.436964 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.436940 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:58:21.456446 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.456409 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" Jul 09 18:58:21.472099 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.472075 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:58:21.473030 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.473017 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" Jul 09 18:58:21.482014 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.482000 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:58:21.936093 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.936056 2561 apiserver.go:52] "Watching apiserver" Jul 09 18:58:21.947245 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.947216 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 18:58:21.948183 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.948145 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal","openshift-multus/network-metrics-daemon-stgpg","openshift-network-diagnostics/network-check-target-89bcl","openshift-network-operator/iptables-alerter-dzdrl","openshift-ovn-kubernetes/ovnkube-node-c6t5s","kube-system/global-pull-secret-syncer-jfmg2","kube-system/konnectivity-agent-4j9q5","kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz","openshift-cluster-node-tuning-operator/tuned-888ll","openshift-image-registry/node-ca-2mbf8","openshift-multus/multus-452qh","openshift-multus/multus-additional-cni-plugins-mdhz8","openshift-dns/node-resolver-rk9t5"] Jul 09 18:58:21.949647 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.949583 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:21.952731 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.952709 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.953214 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.953193 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 18:58:21.953633 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.953614 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gkt44\"" Jul 09 18:58:21.953633 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.953625 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 18:58:21.954226 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.953913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:21.954226 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:21.953986 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:21.955174 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955124 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.955417 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955402 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 18:58:21.955760 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955674 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 18:58:21.955760 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955685 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 18:58:21.955760 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955726 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:21.955955 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955727 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.955955 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.955864 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-65lmx\"" Jul 09 18:58:21.956753 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.956735 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.957665 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.956927 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 18:58:21.958211 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.957951 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:21.958211 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:21.958011 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:21.958211 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.958082 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.958380 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.958297 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ftp8m\"" Jul 09 18:58:21.958414 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.958397 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 18:58:21.959104 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.959083 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.959227 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.959212 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 18:58:21.959416 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.959402 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.959488 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.959442 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 18:58:21.959770 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.959669 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-htlzl\"" Jul 09 18:58:21.959828 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.959782 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 18:58:21.960875 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.960854 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:21.961732 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.961715 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 18:58:21.961950 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.961930 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.963404 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.962784 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:21.964763 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.963850 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.964763 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.964098 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 18:58:21.964763 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.964292 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.964763 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.964490 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jr259\"" Jul 09 18:58:21.964941 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.964855 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:21.964941 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:21.964907 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:21.965203 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.965183 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.965982 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.965524 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.965982 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.965590 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7s27s\"" Jul 09 18:58:21.966120 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.966028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:21.968314 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.967588 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:21.968314 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.967924 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:58:21.968314 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968203 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-cnibin\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.968314 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968232 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.968314 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-env-overrides\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968314 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovnkube-script-lib\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52c8d752-0a4d-4e02-85f9-435185c3da3d-kubelet-config\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968359 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48ebe129-5eff-4908-ab95-f01e4e247026-konnectivity-ca\") pod \"konnectivity-agent-4j9q5\" (UID: \"48ebe129-5eff-4908-ab95-f01e4e247026\") " pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-systemd\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968405 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-etc-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968422 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-cni-bin\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968438 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-system-cni-dir\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968452 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-kubelet\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-systemd-units\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968488 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-slash\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968523 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqfp5\" (UniqueName: \"kubernetes.io/projected/012fe783-7cfb-4569-85c6-b9133d47bc5d-kube-api-access-dqfp5\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmklx\" (UniqueName: \"kubernetes.io/projected/82b32130-4fe5-40fd-8975-6ff46d48d1af-kube-api-access-pmklx\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-var-lib-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968600 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968623 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-log-socket\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.968682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968655 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-os-release\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968716 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sj4v7\"" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968725 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82b32130-4fe5-40fd-8975-6ff46d48d1af-host-slash\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-cni-netd\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968823 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovn-node-metrics-cert\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48ebe129-5eff-4908-ab95-f01e4e247026-agent-certs\") pod \"konnectivity-agent-4j9q5\" (UID: \"48ebe129-5eff-4908-ab95-f01e4e247026\") " pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968883 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovnkube-config\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968937 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968962 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-run-netns\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.968984 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-node-log\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969006 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969010 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969031 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52c8d752-0a4d-4e02-85f9-435185c3da3d-dbus\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969048 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:21.969389 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969068 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82b32130-4fe5-40fd-8975-6ff46d48d1af-iptables-alerter-script\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:21.970232 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969097 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-ovn\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.970232 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969118 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5k6\" (UniqueName: \"kubernetes.io/projected/55d0dba2-5711-485d-a7cb-cfbb268bec42-kube-api-access-fb5k6\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:21.970232 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969137 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:21.970232 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.969278 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.970591 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.970392 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 18:58:21.970649 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.970613 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-svrl4\"" Jul 09 18:58:21.970883 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.970863 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 18:58:21.970965 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.970885 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 18:58:21.971529 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.971290 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-452qh" Jul 09 18:58:21.974475 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.974456 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-krgmb\"" Jul 09 18:58:21.976418 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.974708 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 18:58:21.993523 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.990686 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dqpk2" Jul 09 18:58:21.999013 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:21.998664 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dqpk2" Jul 09 18:58:22.057897 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.057870 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069302 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-kubernetes\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069335 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-socket-dir-parent\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-etc-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-cni-bin\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069387 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdgl8\" (UniqueName: \"kubernetes.io/projected/080589b7-abe1-4464-a9eb-37eb06937242-kube-api-access-jdgl8\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-registration-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-run\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069431 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-kubelet\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-kubelet\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069459 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-systemd-units\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069454 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-etc-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069555 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-kubelet\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069565 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-cni-bin\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069584 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-systemd-units\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069605 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-cnibin\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.069862 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6503b59d-776a-448a-899e-2cd7c73ba171-multus-daemon-config\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqfp5\" (UniqueName: \"kubernetes.io/projected/012fe783-7cfb-4569-85c6-b9133d47bc5d-kube-api-access-dqfp5\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-sys-fs\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069736 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-host\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-cni-netd\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovn-node-metrics-cert\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069877 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-cni-netd\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-socket-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbb9w\" (UniqueName: \"kubernetes.io/projected/6503b59d-776a-448a-899e-2cd7c73ba171-kube-api-access-sbb9w\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.069997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/080589b7-abe1-4464-a9eb-37eb06937242-serviceca\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070022 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-run-netns\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-node-log\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070495 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070115 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-cni-bin\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070194 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070224 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-ovn\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5k6\" (UniqueName: \"kubernetes.io/projected/55d0dba2-5711-485d-a7cb-cfbb268bec42-kube-api-access-fb5k6\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070274 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc2pk\" (UniqueName: \"kubernetes.io/projected/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-kube-api-access-bc2pk\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070326 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8926e4d-69e8-4655-9c00-2532253478e1-tmp\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6503b59d-776a-448a-899e-2cd7c73ba171-cni-binary-copy\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070380 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-env-overrides\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52c8d752-0a4d-4e02-85f9-435185c3da3d-kubelet-config\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070431 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-hosts-file\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48ebe129-5eff-4908-ab95-f01e4e247026-konnectivity-ca\") pod \"konnectivity-agent-4j9q5\" (UID: \"48ebe129-5eff-4908-ab95-f01e4e247026\") " pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070594 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-systemd\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070620 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-device-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-k8s-cni-cncf-io\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070677 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-system-cni-dir\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070703 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-slash\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.070961 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-systemd\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070778 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-sys\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq4f9\" (UniqueName: \"kubernetes.io/projected/b8926e4d-69e8-4655-9c00-2532253478e1-kube-api-access-mq4f9\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmklx\" (UniqueName: \"kubernetes.io/projected/82b32130-4fe5-40fd-8975-6ff46d48d1af-kube-api-access-pmklx\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-var-lib-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070880 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070914 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-log-socket\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070939 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/080589b7-abe1-4464-a9eb-37eb06937242-host\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070962 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-etc-selinux\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.070988 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-var-lib-kubelet\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071013 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-os-release\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82b32130-4fe5-40fd-8975-6ff46d48d1af-host-slash\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.071408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071368 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-cni-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-conf-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071464 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-etc-kubernetes\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48ebe129-5eff-4908-ab95-f01e4e247026-agent-certs\") pod \"konnectivity-agent-4j9q5\" (UID: \"48ebe129-5eff-4908-ab95-f01e4e247026\") " pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071542 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovnkube-config\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4zh\" (UniqueName: \"kubernetes.io/projected/3f59493d-083f-4990-8df2-09f3c8845e86-kube-api-access-sr4zh\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071596 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-modprobe-d\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071588 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-systemd\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071619 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysctl-d\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8926e4d-69e8-4655-9c00-2532253478e1-etc-tuned\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071648 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82b32130-4fe5-40fd-8975-6ff46d48d1af-host-slash\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-netns\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52c8d752-0a4d-4e02-85f9-435185c3da3d-dbus\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysctl-conf\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-lib-modules\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-os-release\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.072015 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071771 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-cni-multus\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071786 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-hostroot\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-multus-certs\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071847 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82b32130-4fe5-40fd-8975-6ff46d48d1af-iptables-alerter-script\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071862 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cq55\" (UniqueName: \"kubernetes.io/projected/94aaa79a-c612-444a-ad52-374b46fa0709-kube-api-access-9cq55\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071878 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-system-cni-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071915 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-cnibin\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071950 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovnkube-script-lib\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-tmp-dir\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.071996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysconfig\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072054 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-system-cni-dir\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072077 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-slash\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072231 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48ebe129-5eff-4908-ab95-f01e4e247026-konnectivity-ca\") pod \"konnectivity-agent-4j9q5\" (UID: \"48ebe129-5eff-4908-ab95-f01e4e247026\") " pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072256 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-var-lib-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072282 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-log-socket\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.074721 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072332 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-os-release\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072578 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-ovn\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072618 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/52c8d752-0a4d-4e02-85f9-435185c3da3d-dbus\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.072740 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073064 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovnkube-config\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073108 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073444 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-env-overrides\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073531 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/52c8d752-0a4d-4e02-85f9-435185c3da3d-kubelet-config\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073729 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/82b32130-4fe5-40fd-8975-6ff46d48d1af-iptables-alerter-script\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073744 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-node-log\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073781 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073818 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012fe783-7cfb-4569-85c6-b9133d47bc5d-cnibin\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.073821 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073850 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-run-openvswitch\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.073883 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55d0dba2-5711-485d-a7cb-cfbb268bec42-host-run-netns\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.073963 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:58:22.573932624 +0000 UTC m=+3.025385062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:22.075307 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.074261 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/012fe783-7cfb-4569-85c6-b9133d47bc5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.076044 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.074272 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovnkube-script-lib\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.076256 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.076236 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55d0dba2-5711-485d-a7cb-cfbb268bec42-ovn-node-metrics-cert\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.077682 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.077658 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48ebe129-5eff-4908-ab95-f01e4e247026-agent-certs\") pod \"konnectivity-agent-4j9q5\" (UID: \"48ebe129-5eff-4908-ab95-f01e4e247026\") " pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:22.078400 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.078354 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqfp5\" (UniqueName: \"kubernetes.io/projected/012fe783-7cfb-4569-85c6-b9133d47bc5d-kube-api-access-dqfp5\") pod \"multus-additional-cni-plugins-mdhz8\" (UID: \"012fe783-7cfb-4569-85c6-b9133d47bc5d\") " pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.079734 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.079711 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:22.079816 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.079741 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:22.079816 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.079758 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:22.079879 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.079825 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:22.579807542 +0000 UTC m=+3.031260026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:22.081063 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.080921 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmklx\" (UniqueName: \"kubernetes.io/projected/82b32130-4fe5-40fd-8975-6ff46d48d1af-kube-api-access-pmklx\") pod \"iptables-alerter-dzdrl\" (UID: \"82b32130-4fe5-40fd-8975-6ff46d48d1af\") " pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.081063 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.081037 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5k6\" (UniqueName: \"kubernetes.io/projected/55d0dba2-5711-485d-a7cb-cfbb268bec42-kube-api-access-fb5k6\") pod \"ovnkube-node-c6t5s\" (UID: \"55d0dba2-5711-485d-a7cb-cfbb268bec42\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.110782 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.110746 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:58:22.172522 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-sys-fs\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172532 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-host\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172559 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-socket-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbb9w\" (UniqueName: \"kubernetes.io/projected/6503b59d-776a-448a-899e-2cd7c73ba171-kube-api-access-sbb9w\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172610 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/080589b7-abe1-4464-a9eb-37eb06937242-serviceca\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-cni-bin\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172650 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc2pk\" (UniqueName: \"kubernetes.io/projected/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-kube-api-access-bc2pk\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172665 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8926e4d-69e8-4655-9c00-2532253478e1-tmp\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.172683 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172678 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6503b59d-776a-448a-899e-2cd7c73ba171-cni-binary-copy\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-hosts-file\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-device-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-k8s-cni-cncf-io\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-systemd\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172801 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-sys\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq4f9\" (UniqueName: \"kubernetes.io/projected/b8926e4d-69e8-4655-9c00-2532253478e1-kube-api-access-mq4f9\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172845 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/080589b7-abe1-4464-a9eb-37eb06937242-host\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-etc-selinux\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172875 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-var-lib-kubelet\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-cni-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172920 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-conf-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172944 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-etc-kubernetes\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172969 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4zh\" (UniqueName: \"kubernetes.io/projected/3f59493d-083f-4990-8df2-09f3c8845e86-kube-api-access-sr4zh\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172984 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-modprobe-d\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.172997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysctl-d\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8926e4d-69e8-4655-9c00-2532253478e1-etc-tuned\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173036 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-netns\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysctl-conf\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-lib-modules\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173116 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-os-release\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-cni-multus\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-hostroot\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173179 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-multus-certs\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cq55\" (UniqueName: \"kubernetes.io/projected/94aaa79a-c612-444a-ad52-374b46fa0709-kube-api-access-9cq55\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173219 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-system-cni-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-tmp-dir\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysconfig\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173294 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-kubernetes\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173313 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-socket-dir-parent\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173331 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdgl8\" (UniqueName: \"kubernetes.io/projected/080589b7-abe1-4464-a9eb-37eb06937242-kube-api-access-jdgl8\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-registration-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-run\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173396 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-kubelet\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.173765 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173446 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-cnibin\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6503b59d-776a-448a-899e-2cd7c73ba171-multus-daemon-config\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173618 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-sys\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173637 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173696 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-k8s-cni-cncf-io\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173748 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-systemd\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173797 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-sys-fs\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173843 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-host\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173898 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/080589b7-abe1-4464-a9eb-37eb06937242-host\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173957 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-socket-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.173962 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-etc-selinux\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174028 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-cni-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174068 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-conf-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.174385 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174110 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6503b59d-776a-448a-899e-2cd7c73ba171-multus-daemon-config\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174405 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/080589b7-abe1-4464-a9eb-37eb06937242-serviceca\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174446 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-hosts-file\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174568 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6503b59d-776a-448a-899e-2cd7c73ba171-cni-binary-copy\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174618 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-cni-bin\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174630 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-var-lib-kubelet\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174704 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-modprobe-d\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174735 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysconfig\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174764 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-kubernetes\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174805 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-multus-socket-dir-parent\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174860 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-tmp-dir\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174959 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysctl-d\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174970 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-registration-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175009 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-run\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175036 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-kubelet\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.175086 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:22.175115 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.175126 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:22.675112501 +0000 UTC m=+3.126564938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-var-lib-cni-multus\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175280 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-cnibin\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175320 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94aaa79a-c612-444a-ad52-374b46fa0709-device-dir\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.174113 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-etc-kubernetes\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-lib-modules\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175352 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8926e4d-69e8-4655-9c00-2532253478e1-etc-sysctl-conf\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175372 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-netns\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175403 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-host-run-multus-certs\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175409 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-os-release\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175452 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-hostroot\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176010 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.175464 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6503b59d-776a-448a-899e-2cd7c73ba171-system-cni-dir\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.176967 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.176927 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8926e4d-69e8-4655-9c00-2532253478e1-tmp\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.177353 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.177335 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8926e4d-69e8-4655-9c00-2532253478e1-etc-tuned\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.184883 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.184857 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc2pk\" (UniqueName: \"kubernetes.io/projected/56b19497-c17c-4c4e-85fc-9e1ceedaa60f-kube-api-access-bc2pk\") pod \"node-resolver-rk9t5\" (UID: \"56b19497-c17c-4c4e-85fc-9e1ceedaa60f\") " pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.185094 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.185073 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cq55\" (UniqueName: \"kubernetes.io/projected/94aaa79a-c612-444a-ad52-374b46fa0709-kube-api-access-9cq55\") pod \"aws-ebs-csi-driver-node-rbdtz\" (UID: \"94aaa79a-c612-444a-ad52-374b46fa0709\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.185143 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.185105 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq4f9\" (UniqueName: \"kubernetes.io/projected/b8926e4d-69e8-4655-9c00-2532253478e1-kube-api-access-mq4f9\") pod \"tuned-888ll\" (UID: \"b8926e4d-69e8-4655-9c00-2532253478e1\") " pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.186448 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.186018 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4zh\" (UniqueName: \"kubernetes.io/projected/3f59493d-083f-4990-8df2-09f3c8845e86-kube-api-access-sr4zh\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:22.186448 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.186379 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbb9w\" (UniqueName: \"kubernetes.io/projected/6503b59d-776a-448a-899e-2cd7c73ba171-kube-api-access-sbb9w\") pod \"multus-452qh\" (UID: \"6503b59d-776a-448a-899e-2cd7c73ba171\") " pod="openshift-multus/multus-452qh" Jul 09 18:58:22.189442 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.189417 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdgl8\" (UniqueName: \"kubernetes.io/projected/080589b7-abe1-4464-a9eb-37eb06937242-kube-api-access-jdgl8\") pod \"node-ca-2mbf8\" (UID: \"080589b7-abe1-4464-a9eb-37eb06937242\") " pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.266515 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.266474 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:22.275473 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.275451 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" Jul 09 18:58:22.284167 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.284138 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dzdrl" Jul 09 18:58:22.295872 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.295844 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:22.301482 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.301463 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" Jul 09 18:58:22.311129 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.311102 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rk9t5" Jul 09 18:58:22.319259 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.319236 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-888ll" Jul 09 18:58:22.326883 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.326864 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2mbf8" Jul 09 18:58:22.334472 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.334450 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-452qh" Jul 09 18:58:22.576748 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.576651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:22.576926 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.576819 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:22.576926 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.576877 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:58:23.576865143 +0000 UTC m=+4.028317580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:22.677773 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.677738 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:22.677773 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:22.677782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:22.678030 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.677911 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:22.678030 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.677945 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:22.678030 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.677962 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:22.678030 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.677975 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:22.678030 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.677984 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:23.677962583 +0000 UTC m=+4.129415030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:22.678030 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:22.678022 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:23.678006023 +0000 UTC m=+4.129458463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:23.002029 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.001946 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:53:21 +0000 UTC" deadline="2028-01-23 05:22:51.652602783 +0000 UTC" Jul 09 18:58:23.002029 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.001981 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13498h24m28.650624624s" Jul 09 18:58:23.049823 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.049791 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ebe129_5eff_4908_ab95_f01e4e247026.slice/crio-fccbdc485dd8dfbd76a7dbdf0ffa26a85fd37c09e0621e3563afa7bbd5b7f53e WatchSource:0}: Error finding container fccbdc485dd8dfbd76a7dbdf0ffa26a85fd37c09e0621e3563afa7bbd5b7f53e: Status 404 returned error can't find the container with id fccbdc485dd8dfbd76a7dbdf0ffa26a85fd37c09e0621e3563afa7bbd5b7f53e Jul 09 18:58:23.050452 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.050241 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d0dba2_5711_485d_a7cb_cfbb268bec42.slice/crio-5a6525c026e2c9b4a030bf66a64cb47a03b1f00d3c57351ad5c26dd459582b62 WatchSource:0}: Error finding container 5a6525c026e2c9b4a030bf66a64cb47a03b1f00d3c57351ad5c26dd459582b62: Status 404 returned error can't find the container with id 5a6525c026e2c9b4a030bf66a64cb47a03b1f00d3c57351ad5c26dd459582b62 Jul 09 18:58:23.051623 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.051604 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b32130_4fe5_40fd_8975_6ff46d48d1af.slice/crio-bacce1552f48d0d993254866d85387f231df86aace236809fdf8fffb02683f49 WatchSource:0}: Error finding container bacce1552f48d0d993254866d85387f231df86aace236809fdf8fffb02683f49: Status 404 returned error can't find the container with id bacce1552f48d0d993254866d85387f231df86aace236809fdf8fffb02683f49 Jul 09 18:58:23.053047 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.053024 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8926e4d_69e8_4655_9c00_2532253478e1.slice/crio-c6e6c327ff8f7831204672ed9362dc94699172f980d94000d1800a955ef6b7b0 WatchSource:0}: Error finding container c6e6c327ff8f7831204672ed9362dc94699172f980d94000d1800a955ef6b7b0: Status 404 returned error can't find the container with id c6e6c327ff8f7831204672ed9362dc94699172f980d94000d1800a955ef6b7b0 Jul 09 18:58:23.053938 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.053804 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6503b59d_776a_448a_899e_2cd7c73ba171.slice/crio-54826207ad4019edbdae6f6c6690b8e5399967e7beb13657e93ed4f7da8274d7 WatchSource:0}: Error finding container 54826207ad4019edbdae6f6c6690b8e5399967e7beb13657e93ed4f7da8274d7: Status 404 returned error can't find the container with id 54826207ad4019edbdae6f6c6690b8e5399967e7beb13657e93ed4f7da8274d7 Jul 09 18:58:23.059534 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.059489 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b19497_c17c_4c4e_85fc_9e1ceedaa60f.slice/crio-4ba2717a48f8772ce1cf7f40cedcc71c1491acc4dfa37c64c86005396cc88028 WatchSource:0}: Error finding container 4ba2717a48f8772ce1cf7f40cedcc71c1491acc4dfa37c64c86005396cc88028: Status 404 returned error can't find the container with id 4ba2717a48f8772ce1cf7f40cedcc71c1491acc4dfa37c64c86005396cc88028 Jul 09 18:58:23.060569 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.060544 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080589b7_abe1_4464_a9eb_37eb06937242.slice/crio-2b265a3eff9e541d961bb67cde59883befb69b6997a0a99593a2870b6ca066d5 WatchSource:0}: Error finding container 2b265a3eff9e541d961bb67cde59883befb69b6997a0a99593a2870b6ca066d5: Status 404 returned error can't find the container with id 2b265a3eff9e541d961bb67cde59883befb69b6997a0a99593a2870b6ca066d5 Jul 09 18:58:23.062366 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:23.062341 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94aaa79a_c612_444a_ad52_374b46fa0709.slice/crio-925c63e481a28c312e9de0cba57e5ec265d7b457c89cfaa378a025ce2e99f4fc WatchSource:0}: Error finding container 925c63e481a28c312e9de0cba57e5ec265d7b457c89cfaa378a025ce2e99f4fc: Status 404 returned error can't find the container with id 925c63e481a28c312e9de0cba57e5ec265d7b457c89cfaa378a025ce2e99f4fc Jul 09 18:58:23.104775 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.104742 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" event={"ID":"94aaa79a-c612-444a-ad52-374b46fa0709","Type":"ContainerStarted","Data":"925c63e481a28c312e9de0cba57e5ec265d7b457c89cfaa378a025ce2e99f4fc"} Jul 09 18:58:23.107080 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.107053 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2mbf8" event={"ID":"080589b7-abe1-4464-a9eb-37eb06937242","Type":"ContainerStarted","Data":"2b265a3eff9e541d961bb67cde59883befb69b6997a0a99593a2870b6ca066d5"} Jul 09 18:58:23.108128 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.108107 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-888ll" event={"ID":"b8926e4d-69e8-4655-9c00-2532253478e1","Type":"ContainerStarted","Data":"c6e6c327ff8f7831204672ed9362dc94699172f980d94000d1800a955ef6b7b0"} Jul 09 18:58:23.109181 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.109152 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"5a6525c026e2c9b4a030bf66a64cb47a03b1f00d3c57351ad5c26dd459582b62"} Jul 09 18:58:23.110088 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.110070 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dzdrl" event={"ID":"82b32130-4fe5-40fd-8975-6ff46d48d1af","Type":"ContainerStarted","Data":"bacce1552f48d0d993254866d85387f231df86aace236809fdf8fffb02683f49"} Jul 09 18:58:23.111154 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.111129 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4j9q5" event={"ID":"48ebe129-5eff-4908-ab95-f01e4e247026","Type":"ContainerStarted","Data":"fccbdc485dd8dfbd76a7dbdf0ffa26a85fd37c09e0621e3563afa7bbd5b7f53e"} Jul 09 18:58:23.112282 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.112257 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rk9t5" event={"ID":"56b19497-c17c-4c4e-85fc-9e1ceedaa60f","Type":"ContainerStarted","Data":"4ba2717a48f8772ce1cf7f40cedcc71c1491acc4dfa37c64c86005396cc88028"} Jul 09 18:58:23.113320 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.113297 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerStarted","Data":"9d102cb3ba6306ed76b875803877007db03c7a768dde6f0721d57d37d5aaca47"} Jul 09 18:58:23.114289 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.114270 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-452qh" event={"ID":"6503b59d-776a-448a-899e-2cd7c73ba171","Type":"ContainerStarted","Data":"54826207ad4019edbdae6f6c6690b8e5399967e7beb13657e93ed4f7da8274d7"} Jul 09 18:58:23.584654 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.584399 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:23.584867 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.584790 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:23.584867 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.584853 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:58:25.584835534 +0000 UTC m=+6.036287976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:23.685903 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.685860 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:23.686093 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:23.685924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:23.686158 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.686122 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:23.686158 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.686141 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:23.686158 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.686154 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:23.686306 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.686213 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:25.686193438 +0000 UTC m=+6.137645896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:23.686832 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.686810 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:23.686951 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:23.686902 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:25.686884217 +0000 UTC m=+6.138336660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:24.003116 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.002943 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:53:21 +0000 UTC" deadline="2028-03-20 20:43:32.93231832 +0000 UTC" Jul 09 18:58:24.003116 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.003033 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14881h45m8.929290749s" Jul 09 18:58:24.094702 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.094637 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:24.094702 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.094658 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:24.094702 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.094681 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:24.095010 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:24.094772 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:24.097371 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:24.095203 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:24.097371 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:24.095297 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:24.134280 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.134238 2561 generic.go:356] "Generic (PLEG): container finished" podID="752495a5e2997f2598ef0fe538e1c710" containerID="ec7d75e50e971e39569cce60226cbd3a67b128991776ea25c1072005e34dfbbc" exitCode=0 Jul 09 18:58:24.134491 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.134360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" event={"ID":"752495a5e2997f2598ef0fe538e1c710","Type":"ContainerDied","Data":"ec7d75e50e971e39569cce60226cbd3a67b128991776ea25c1072005e34dfbbc"} Jul 09 18:58:24.151016 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:24.150974 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" event={"ID":"2167be909be366b8f502c0ed2d1d4ffd","Type":"ContainerStarted","Data":"1bc7027de031d0053272815ebf7c595e6595e75eeea40431319a6917e53ed2f0"} Jul 09 18:58:25.182521 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:25.182473 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" event={"ID":"752495a5e2997f2598ef0fe538e1c710","Type":"ContainerStarted","Data":"28198c24d6cb686a23efc71d29843e18831f95fcf6638d4c10bf12e03133be86"} Jul 09 18:58:25.197535 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:25.197230 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-17.ec2.internal" podStartSLOduration=4.197207406 podStartE2EDuration="4.197207406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:24.172385499 +0000 UTC m=+4.623837961" watchObservedRunningTime="2026-07-09 18:58:25.197207406 +0000 UTC m=+5.648659867" Jul 09 18:58:25.197829 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:25.197788 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-17.ec2.internal" podStartSLOduration=4.197778108 podStartE2EDuration="4.197778108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:25.196730645 +0000 UTC m=+5.648183106" watchObservedRunningTime="2026-07-09 18:58:25.197778108 +0000 UTC m=+5.649230568" Jul 09 18:58:25.607273 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:25.607189 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:25.607422 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.607376 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:25.607481 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.607438 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:58:29.607419465 +0000 UTC m=+10.058871906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:25.707918 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:25.707879 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:25.708158 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:25.707987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:25.708158 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.708104 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:25.708158 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.708160 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:29.708141674 +0000 UTC m=+10.159594119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:25.708591 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.708571 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:25.708687 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.708601 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:25.708687 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.708615 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:25.708687 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:25.708659 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:29.708644752 +0000 UTC m=+10.160097195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:26.096624 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:26.096537 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:26.096777 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:26.096679 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:26.097093 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:26.097075 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:26.097357 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:26.097167 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:26.097357 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:26.097219 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:26.097357 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:26.097290 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:28.096815 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:28.096779 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:28.097255 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:28.096894 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:28.097255 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:28.097189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:28.097255 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:28.097245 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:28.097357 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:28.097285 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:28.097357 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:28.097332 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:29.642986 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:29.642938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:29.643614 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.643097 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:29.643614 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.643168 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:58:37.643147364 +0000 UTC m=+18.094599801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:29.743406 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:29.743372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:29.743629 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:29.743425 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:29.743695 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.743631 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:29.743695 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.743648 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:29.743695 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.743658 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:29.743849 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.743709 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:37.743695271 +0000 UTC m=+18.195147709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:29.744226 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.744199 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:29.744353 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:29.744307 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:37.744290095 +0000 UTC m=+18.195742538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:30.097040 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:30.096611 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:30.097262 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:30.097235 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:30.097365 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:30.097342 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:30.097564 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:30.097530 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:30.097706 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:30.097645 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:30.097763 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:30.097700 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:32.094752 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:32.094714 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:32.095244 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:32.094784 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:32.095244 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:32.094798 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:32.095244 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:32.094882 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:32.095244 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:32.094961 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:32.095244 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:32.095021 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:34.095150 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:34.095098 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:34.095865 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:34.095228 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:34.095865 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:34.095098 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:34.095865 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:34.095344 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:34.095865 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:34.095382 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:34.095865 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:34.095439 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:36.094905 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:36.094881 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:36.094905 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:36.094893 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:36.095372 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:36.094882 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:36.095372 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:36.094999 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:36.095372 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:36.095185 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:36.095372 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:36.095257 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:37.707742 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:37.707704 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:37.708181 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.707883 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:37.708181 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.707961 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:58:53.707940096 +0000 UTC m=+34.159392552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:37.808674 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:37.808619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:37.808870 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:37.808690 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:37.808870 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.808810 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:37.808984 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.808881 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:53.808861827 +0000 UTC m=+34.260314278 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:37.809215 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.808816 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:37.809301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.809229 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:37.809301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.809251 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:37.809301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:37.809298 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:53.809284101 +0000 UTC m=+34.260736553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:38.095179 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:38.095137 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:38.095375 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:38.095148 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:38.095375 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:38.095277 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:38.095375 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:38.095147 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:38.095375 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:38.095343 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:38.095601 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:38.095444 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:40.095132 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:40.095092 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:40.095636 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:40.095201 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:40.095636 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:40.095276 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:40.095636 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:40.095382 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:40.095636 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:40.095423 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:40.095636 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:40.095496 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:42.094940 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.094764 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:42.095315 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.094766 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:42.095315 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:42.095049 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:42.095315 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:42.095148 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:42.095315 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.094816 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:42.095315 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:42.095258 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:42.216744 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.216716 2561 generic.go:356] "Generic (PLEG): container finished" podID="012fe783-7cfb-4569-85c6-b9133d47bc5d" containerID="48a7704a917d41027459d00b97a9694208289dee4fc94d167cdfe8f024ab7a6e" exitCode=0 Jul 09 18:58:42.216861 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.216794 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerDied","Data":"48a7704a917d41027459d00b97a9694208289dee4fc94d167cdfe8f024ab7a6e"} Jul 09 18:58:42.219040 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.218938 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-452qh" event={"ID":"6503b59d-776a-448a-899e-2cd7c73ba171","Type":"ContainerStarted","Data":"c86184b1a97c45bbbcf8c1ac8c71256c91b61654f44f596667aa8363874a6bbf"} Jul 09 18:58:42.220563 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.220536 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" event={"ID":"94aaa79a-c612-444a-ad52-374b46fa0709","Type":"ContainerStarted","Data":"fe7cbad9146ca7b58bf11c3b544dc153776afc50d79fb36a78bc5024e495dd50"} Jul 09 18:58:42.224547 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.221888 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2mbf8" event={"ID":"080589b7-abe1-4464-a9eb-37eb06937242","Type":"ContainerStarted","Data":"62a1f5bd96dffa7326823a2e5197f74fa3a5144cdb3e9cc278a1fc5e98caa072"} Jul 09 18:58:42.229296 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.229253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-888ll" event={"ID":"b8926e4d-69e8-4655-9c00-2532253478e1","Type":"ContainerStarted","Data":"9970616ef726086f46eabc37b03adf82d1503928e992f42bba5acac14a2631da"} Jul 09 18:58:42.231398 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.231373 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 18:58:42.231735 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.231708 2561 generic.go:356] "Generic (PLEG): container finished" podID="55d0dba2-5711-485d-a7cb-cfbb268bec42" containerID="8dbd1aedd902c5e6ee4293395cf509151f3748542ee1299373418239b88ac159" exitCode=1 Jul 09 18:58:42.231808 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.231781 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"60e7eb8133019dc8d1f65c6bfdc53239e6a91ad170eced6d05a4eba69b3cb4e6"} Jul 09 18:58:42.231874 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.231860 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerDied","Data":"8dbd1aedd902c5e6ee4293395cf509151f3748542ee1299373418239b88ac159"} Jul 09 18:58:42.231941 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.231905 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"fcda293461f529c13d58aadee2cbb115451c21303c8360749e1587ef0cac385d"} Jul 09 18:58:42.233229 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.233203 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4j9q5" event={"ID":"48ebe129-5eff-4908-ab95-f01e4e247026","Type":"ContainerStarted","Data":"38a6dfcc86bf491a736a364189133de23b09804b8cc5cc89ca9142ced230c734"} Jul 09 18:58:42.234666 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.234643 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rk9t5" event={"ID":"56b19497-c17c-4c4e-85fc-9e1ceedaa60f","Type":"ContainerStarted","Data":"560d4911ad705be176c15e85fe6fb2c5e23b2dcd43edcc416472d5efb7a7bf10"} Jul 09 18:58:42.259500 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.259451 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rk9t5" podStartSLOduration=3.51524841 podStartE2EDuration="22.259432465s" totalImagesPullingTime="18.744184055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:42.259121862 +0000 UTC m=+22.710574323" watchObservedRunningTime="2026-07-09 18:58:42.259432465 +0000 UTC m=+22.710884924" Jul 09 18:58:42.276546 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.276488 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-888ll" podStartSLOduration=3.526430122 podStartE2EDuration="22.276470125s" totalImagesPullingTime="18.750040003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:42.276447914 +0000 UTC m=+22.727900374" watchObservedRunningTime="2026-07-09 18:58:42.276470125 +0000 UTC m=+22.727922586" Jul 09 18:58:42.290689 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.290641 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2mbf8" podStartSLOduration=3.546432648 podStartE2EDuration="22.290622027s" totalImagesPullingTime="18.744189379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:42.290372495 +0000 UTC m=+22.741824954" watchObservedRunningTime="2026-07-09 18:58:42.290622027 +0000 UTC m=+22.742074489" Jul 09 18:58:42.334467 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.334421 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4j9q5" podStartSLOduration=3.580744565 podStartE2EDuration="22.334400892s" totalImagesPullingTime="18.753656327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:42.334100308 +0000 UTC m=+22.785552768" watchObservedRunningTime="2026-07-09 18:58:42.334400892 +0000 UTC m=+22.785853356" Jul 09 18:58:42.334698 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:42.334672 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-452qh" podStartSLOduration=3.520315941 podStartE2EDuration="22.334666147s" totalImagesPullingTime="18.814350206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:42.315610725 +0000 UTC m=+22.767063189" watchObservedRunningTime="2026-07-09 18:58:42.334666147 +0000 UTC m=+22.786118606" Jul 09 18:58:43.240298 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.240266 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 18:58:43.241129 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.240675 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"c566ff7eda9cf476f2d5885e59c803f8feaf395798e07630c9848805bff5fffc"} Jul 09 18:58:43.241129 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.240766 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"b947693b31191a975ec34b27a4708b34c8d430809ee4032c77c5579670683533"} Jul 09 18:58:43.241129 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.240828 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"6ab854bd1d0e3ab82d685ebebf9a0fef47b5fdd2dd7f9d104f420a956c3fdea9"} Jul 09 18:58:43.242406 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.242378 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dzdrl" event={"ID":"82b32130-4fe5-40fd-8975-6ff46d48d1af","Type":"ContainerStarted","Data":"39f0bbc77baeb56d9e75ddaa6756c736ffa379cd75b5f9394587db6e230f5aed"} Jul 09 18:58:43.257472 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.257420 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dzdrl" podStartSLOduration=4.504408399 podStartE2EDuration="23.257401888s" totalImagesPullingTime="18.752993489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:43.256745104 +0000 UTC m=+23.708197564" watchObservedRunningTime="2026-07-09 18:58:43.257401888 +0000 UTC m=+23.708854351" Jul 09 18:58:43.402738 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.402710 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 18:58:43.865433 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.865391 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:43.866066 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:43.866048 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:44.035097 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.034947 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:58:43.402732946Z","UUID":"37aa0317-abd9-43b3-bed2-1e1b3ecbc569","Handler":null,"Name":"","Endpoint":""} Jul 09 18:58:44.036484 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.036464 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 18:58:44.036624 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.036519 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 18:58:44.095611 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.095578 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:44.095762 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.095596 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:44.095762 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:44.095697 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:44.095762 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:44.095741 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:44.095874 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.095793 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:44.095874 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:44.095866 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:44.246816 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:44.246781 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" event={"ID":"94aaa79a-c612-444a-ad52-374b46fa0709","Type":"ContainerStarted","Data":"8c5fdb52349f15552381f727b27ac5a7b04f125b48669bbac3bef60163cd5930"} Jul 09 18:58:45.251113 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:45.250913 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" event={"ID":"94aaa79a-c612-444a-ad52-374b46fa0709","Type":"ContainerStarted","Data":"9699bfb4d4a09ce8f6e6cfc658c4a5b75abbaf5c5d11c112ee18282bc9bcd545"} Jul 09 18:58:45.253928 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:45.253908 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 18:58:45.254282 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:45.254266 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 18:58:45.254369 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:45.254265 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"805b8865e59b3b7fd4aabfe2327e85b96a6713a4b479feae69d6af88c12b2300"} Jul 09 18:58:45.269772 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:45.269723 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbdtz" podStartSLOduration=4.055042099 podStartE2EDuration="25.269708885s" totalImagesPullingTime="21.214666786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:45.269253589 +0000 UTC m=+25.720706063" watchObservedRunningTime="2026-07-09 18:58:45.269708885 +0000 UTC m=+25.721161344" Jul 09 18:58:46.094636 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:46.094605 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:46.094636 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:46.094643 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:46.094871 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:46.094620 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:46.094871 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:46.094735 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:46.094871 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:46.094774 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:46.094871 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:46.094837 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:48.094546 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.094522 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:48.094546 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.094535 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:48.094962 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.094531 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:48.094962 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:48.094615 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:48.094962 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:48.094692 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:48.094962 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:48.094767 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:48.265092 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.264946 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 18:58:48.265465 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.265444 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"d3a72d14d3cf036bd6b5c4ab8931ffa07c51e77ee6eeb791c78835eda4d73516"} Jul 09 18:58:48.265753 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.265731 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:48.265870 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.265807 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:48.265927 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.265893 2561 scope.go:117] "RemoveContainer" containerID="8dbd1aedd902c5e6ee4293395cf509151f3748542ee1299373418239b88ac159" Jul 09 18:58:48.267258 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.267236 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerStarted","Data":"2d6e3c0ffc34d0b3a0e4b6d2507b69855ca1d3d6ca414ca004d30b54157fb28d"} Jul 09 18:58:48.283018 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:48.283001 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:49.271665 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.271626 2561 generic.go:356] "Generic (PLEG): container finished" podID="012fe783-7cfb-4569-85c6-b9133d47bc5d" containerID="2d6e3c0ffc34d0b3a0e4b6d2507b69855ca1d3d6ca414ca004d30b54157fb28d" exitCode=0 Jul 09 18:58:49.272303 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.271734 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerDied","Data":"2d6e3c0ffc34d0b3a0e4b6d2507b69855ca1d3d6ca414ca004d30b54157fb28d"} Jul 09 18:58:49.276622 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.276604 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 18:58:49.276989 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.276965 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" event={"ID":"55d0dba2-5711-485d-a7cb-cfbb268bec42","Type":"ContainerStarted","Data":"da813b87d76e40b13c4ffc7feef8f168921db112a992561fbe24f3c393a53277"} Jul 09 18:58:49.277324 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.277297 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:49.293262 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.293235 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:58:49.320033 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:49.319983 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" podStartSLOduration=10.496322226 podStartE2EDuration="29.319959901s" totalImagesPullingTime="18.823637675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:49.31984027 +0000 UTC m=+29.771292730" watchObservedRunningTime="2026-07-09 18:58:49.319959901 +0000 UTC m=+29.771412360" Jul 09 18:58:50.100191 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.100159 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:50.100392 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:50.100266 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:50.100658 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.100641 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:50.100761 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:50.100728 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:50.100881 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.100808 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:50.100953 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:50.100933 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:50.101134 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.101117 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-89bcl"] Jul 09 18:58:50.104358 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.104334 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-stgpg"] Jul 09 18:58:50.105021 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.105002 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jfmg2"] Jul 09 18:58:50.279444 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.279406 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:50.279444 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.279443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:50.280193 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:50.279572 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:50.280193 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:50.279682 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:50.280811 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:50.280791 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:50.280973 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:50.280899 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:51.283805 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:51.283771 2561 generic.go:356] "Generic (PLEG): container finished" podID="012fe783-7cfb-4569-85c6-b9133d47bc5d" containerID="96e5a4cfe6f956fa16a130032a30c7ddf9c8f882d9c8e87842995bb01e616783" exitCode=0 Jul 09 18:58:51.284282 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:51.283855 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerDied","Data":"96e5a4cfe6f956fa16a130032a30c7ddf9c8f882d9c8e87842995bb01e616783"} Jul 09 18:58:51.531890 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:51.531860 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:51.532063 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:51.531988 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 18:58:51.532559 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:51.532537 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4j9q5" Jul 09 18:58:52.094775 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:52.094590 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:52.094958 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:52.094603 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:52.094958 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:52.094877 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:52.094958 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:52.094621 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:52.095102 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:52.094924 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:52.095156 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:52.095101 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:53.290344 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:53.290311 2561 generic.go:356] "Generic (PLEG): container finished" podID="012fe783-7cfb-4569-85c6-b9133d47bc5d" containerID="f0d22a2422c3a8bb540e73186e7f4776ddff471a9122e2a54ad2e8465adb3073" exitCode=0 Jul 09 18:58:53.290832 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:53.290352 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerDied","Data":"f0d22a2422c3a8bb540e73186e7f4776ddff471a9122e2a54ad2e8465adb3073"} Jul 09 18:58:53.716758 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:53.716660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:53.716904 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.716801 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:53.716904 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.716856 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret podName:52c8d752-0a4d-4e02-85f9-435185c3da3d nodeName:}" failed. No retries permitted until 2026-07-09 18:59:25.71684018 +0000 UTC m=+66.168292618 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret") pod "global-pull-secret-syncer-jfmg2" (UID: "52c8d752-0a4d-4e02-85f9-435185c3da3d") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:58:53.817726 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:53.817688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:53.817726 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:53.817732 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:53.817938 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.817857 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:58:53.817938 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.817857 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:53.817938 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.817871 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:58:53.817938 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.817885 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kxqbb for pod openshift-network-diagnostics/network-check-target-89bcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:53.817938 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.817924 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:25.817908525 +0000 UTC m=+66.269360964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:58:53.817938 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:53.817939 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb podName:a83ba2d3-877c-47dd-a2f4-b85ae700a975 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:25.817932303 +0000 UTC m=+66.269384741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxqbb" (UniqueName: "kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb") pod "network-check-target-89bcl" (UID: "a83ba2d3-877c-47dd-a2f4-b85ae700a975") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:58:54.094662 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.094626 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:54.094853 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.094631 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:54.094853 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.094745 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:54.094853 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:54.094740 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-89bcl" podUID="a83ba2d3-877c-47dd-a2f4-b85ae700a975" Jul 09 18:58:54.094853 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:54.094815 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 18:58:54.095063 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:54.094874 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jfmg2" podUID="52c8d752-0a4d-4e02-85f9-435185c3da3d" Jul 09 18:58:54.881168 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.881131 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-17.ec2.internal" event="NodeReady" Jul 09 18:58:54.881581 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.881271 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 18:58:54.914620 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.914586 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-btrnl"] Jul 09 18:58:54.917836 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.917807 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:54.918074 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.918051 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f55989c55-9k6sd"] Jul 09 18:58:54.920472 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.920444 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 18:58:54.920594 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.920480 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 18:58:54.920594 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.920481 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9x2bv\"" Jul 09 18:58:54.921553 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.921533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:54.924153 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.924135 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 18:58:54.924375 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.924337 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 18:58:54.924375 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.924350 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xh4zm\"" Jul 09 18:58:54.924543 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.924399 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 18:58:54.927672 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.927647 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mc95l"] Jul 09 18:58:54.934302 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.934270 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-btrnl"] Jul 09 18:58:54.934409 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.934351 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dkchf"] Jul 09 18:58:54.934597 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.934579 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:54.936864 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.936844 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 18:58:54.937764 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.937491 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bw6lk\"" Jul 09 18:58:54.937764 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.937534 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 18:58:54.937764 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.937573 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 18:58:54.938618 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.938600 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f55989c55-9k6sd"] Jul 09 18:58:54.938726 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.938708 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:54.940007 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.939991 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mc95l"] Jul 09 18:58:54.941557 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.941536 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 18:58:54.941648 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.941536 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 18:58:54.941714 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.941672 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 18:58:54.941767 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.941747 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-65b7z\"" Jul 09 18:58:54.945605 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:54.945587 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dkchf"] Jul 09 18:58:55.025060 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025023 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-bound-sa-token\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025241 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025078 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025241 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025100 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vx6h\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-kube-api-access-8vx6h\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025241 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-trusted-ca\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025241 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-image-registry-private-configuration\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025241 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:55.025241 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025222 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h2sn\" (UniqueName: \"kubernetes.io/projected/8f49887e-0428-4857-8f47-72d48641cea6-kube-api-access-8h2sn\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025253 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-nginx-conf\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025305 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f49887e-0428-4857-8f47-72d48641cea6-tmp-dir\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-installation-pull-secrets\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f7mr\" (UniqueName: \"kubernetes.io/projected/6736da78-2995-4f4b-b579-543e73b3fbd3-kube-api-access-4f7mr\") pod \"ingress-canary-dkchf\" (UID: \"6736da78-2995-4f4b-b579-543e73b3fbd3\") " pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025372 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9442778-7848-4690-a25a-882cf2c57aaf-ca-trust-extracted\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-registry-certificates\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.025541 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025443 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f49887e-0428-4857-8f47-72d48641cea6-config-volume\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.025841 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.025545 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6736da78-2995-4f4b-b579-543e73b3fbd3-cert\") pod \"ingress-canary-dkchf\" (UID: \"6736da78-2995-4f4b-b579-543e73b3fbd3\") " pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.126105 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126066 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-bound-sa-token\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.126105 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.126339 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126122 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vx6h\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-kube-api-access-8vx6h\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.126339 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126243 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:58:55.126339 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126265 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:58:55.126470 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-trusted-ca\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.126470 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126357 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:58:55.626333259 +0000 UTC m=+36.077785720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:58:55.126470 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-image-registry-private-configuration\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.126470 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126426 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:55.126470 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126450 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h2sn\" (UniqueName: \"kubernetes.io/projected/8f49887e-0428-4857-8f47-72d48641cea6-kube-api-access-8h2sn\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-nginx-conf\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126564 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126571 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126629 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:55.626609521 +0000 UTC m=+36.078061959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126664 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126665 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f49887e-0428-4857-8f47-72d48641cea6-tmp-dir\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-installation-pull-secrets\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.126741 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.126718 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:55.626700628 +0000 UTC m=+36.078153080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:58:55.127119 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126756 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f7mr\" (UniqueName: \"kubernetes.io/projected/6736da78-2995-4f4b-b579-543e73b3fbd3-kube-api-access-4f7mr\") pod \"ingress-canary-dkchf\" (UID: \"6736da78-2995-4f4b-b579-543e73b3fbd3\") " pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.127119 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9442778-7848-4690-a25a-882cf2c57aaf-ca-trust-extracted\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.127119 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126835 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-registry-certificates\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.127119 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.126864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f49887e-0428-4857-8f47-72d48641cea6-config-volume\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.127119 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127080 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f49887e-0428-4857-8f47-72d48641cea6-tmp-dir\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.127481 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6736da78-2995-4f4b-b579-543e73b3fbd3-cert\") pod \"ingress-canary-dkchf\" (UID: \"6736da78-2995-4f4b-b579-543e73b3fbd3\") " pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.127481 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127393 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-nginx-conf\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:55.127615 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127486 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f49887e-0428-4857-8f47-72d48641cea6-config-volume\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.127615 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127525 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9442778-7848-4690-a25a-882cf2c57aaf-ca-trust-extracted\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.127857 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127829 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-registry-certificates\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.127986 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.127927 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-trusted-ca\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.131074 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.131051 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-image-registry-private-configuration\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.131154 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.131058 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6736da78-2995-4f4b-b579-543e73b3fbd3-cert\") pod \"ingress-canary-dkchf\" (UID: \"6736da78-2995-4f4b-b579-543e73b3fbd3\") " pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.131154 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.131095 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-installation-pull-secrets\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.134715 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.134667 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h2sn\" (UniqueName: \"kubernetes.io/projected/8f49887e-0428-4857-8f47-72d48641cea6-kube-api-access-8h2sn\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.135024 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.135005 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-bound-sa-token\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.135522 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.135480 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f7mr\" (UniqueName: \"kubernetes.io/projected/6736da78-2995-4f4b-b579-543e73b3fbd3-kube-api-access-4f7mr\") pod \"ingress-canary-dkchf\" (UID: \"6736da78-2995-4f4b-b579-543e73b3fbd3\") " pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.135609 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.135492 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vx6h\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-kube-api-access-8vx6h\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.254092 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.254055 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dkchf" Jul 09 18:58:55.403208 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.403142 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dkchf"] Jul 09 18:58:55.407046 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:58:55.406899 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6736da78_2995_4f4b_b579_543e73b3fbd3.slice/crio-cc6aa03296c0886a48a8728f8c6b4452b2ee5b7adb0ba889cc7a3a82c656c0d6 WatchSource:0}: Error finding container cc6aa03296c0886a48a8728f8c6b4452b2ee5b7adb0ba889cc7a3a82c656c0d6: Status 404 returned error can't find the container with id cc6aa03296c0886a48a8728f8c6b4452b2ee5b7adb0ba889cc7a3a82c656c0d6 Jul 09 18:58:55.630477 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.630438 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:55.630667 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.630539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:55.630667 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.630580 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:55.630667 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630622 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:58:55.630797 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630678 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:58:55.630797 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630683 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:58:55.630797 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630698 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:58:55.630797 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630701 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:56.630680927 +0000 UTC m=+37.082133386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:58:55.630797 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630728 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:56.630712897 +0000 UTC m=+37.082165359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:58:55.630797 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:55.630746 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:58:56.630736464 +0000 UTC m=+37.082188902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:58:55.778350 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.778316 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:55.876967 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.876935 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:55.958673 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:55.958643 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:56.095097 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.095009 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:58:56.095097 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.095073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:58:56.095097 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.095082 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:58:56.099664 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.099637 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:58:56.100045 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.099878 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:58:56.100045 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.099925 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:58:56.100045 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.099945 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rzlbw\"" Jul 09 18:58:56.100045 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.100024 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4jjw\"" Jul 09 18:58:56.100045 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.100049 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:58:56.301225 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.301177 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dkchf" event={"ID":"6736da78-2995-4f4b-b579-543e73b3fbd3","Type":"ContainerStarted","Data":"cc6aa03296c0886a48a8728f8c6b4452b2ee5b7adb0ba889cc7a3a82c656c0d6"} Jul 09 18:58:56.639877 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.639837 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:56.640071 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.639900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:56.640071 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:56.639974 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:56.640071 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640004 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:58:56.640071 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640069 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:58.640050161 +0000 UTC m=+39.091502621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:58:56.640301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640078 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:58:56.640301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640090 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:58:56.640301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640131 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:58:58.640121068 +0000 UTC m=+39.091573506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:58:56.640301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640178 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:58:56.640301 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:56.640260 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:58:58.640241946 +0000 UTC m=+39.091694386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:58:58.657599 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:58.657552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:58.657635 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:58.657697 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657710 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657785 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657790 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657808 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657796 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:02.657773637 +0000 UTC m=+43.109226076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657850 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:02.657836427 +0000 UTC m=+43.109288868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:58:58.658153 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:58:58.657867 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:59:02.657861007 +0000 UTC m=+43.109313445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:58:59.961068 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:58:59.961028 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:00.013968 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:00.013938 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:00.312323 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:00.312288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dkchf" event={"ID":"6736da78-2995-4f4b-b579-543e73b3fbd3","Type":"ContainerStarted","Data":"660c1d851670fbef05114608adcd8a3e1014b8f43efbec665f3c347554602a3d"} Jul 09 18:59:00.315040 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:00.315015 2561 generic.go:356] "Generic (PLEG): container finished" podID="012fe783-7cfb-4569-85c6-b9133d47bc5d" containerID="baafa6c61ad3cc8dcc9369a7bc8f442f3d91a01442b8059fb14458ee42f90db4" exitCode=0 Jul 09 18:59:00.315162 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:00.315058 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerDied","Data":"baafa6c61ad3cc8dcc9369a7bc8f442f3d91a01442b8059fb14458ee42f90db4"} Jul 09 18:59:00.327841 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:00.327798 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dkchf" podStartSLOduration=1.9221900760000001 podStartE2EDuration="6.327783175s" totalImagesPullingTime="4.405593099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:00.327174563 +0000 UTC m=+40.778627024" watchObservedRunningTime="2026-07-09 18:59:00.327783175 +0000 UTC m=+40.779235641" Jul 09 18:59:01.320642 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:01.320607 2561 generic.go:356] "Generic (PLEG): container finished" podID="012fe783-7cfb-4569-85c6-b9133d47bc5d" containerID="2838dcd7570f944530ffe1a9ce7dd99b7805cacd1966288e13eea7ee2ea543ea" exitCode=0 Jul 09 18:59:01.321006 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:01.320697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerDied","Data":"2838dcd7570f944530ffe1a9ce7dd99b7805cacd1966288e13eea7ee2ea543ea"} Jul 09 18:59:02.214348 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.214312 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:02.294230 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.294190 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:02.326614 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.326582 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" event={"ID":"012fe783-7cfb-4569-85c6-b9133d47bc5d","Type":"ContainerStarted","Data":"f0acd93f760d0c069f5e352b4980df343b32fa0dcabb1a74fe10b30542d421cf"} Jul 09 18:59:02.357554 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.357482 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mdhz8" podStartSLOduration=8.750578522 podStartE2EDuration="42.357465932s" totalImagesPullingTime="33.60688741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:02.355433026 +0000 UTC m=+42.806885485" watchObservedRunningTime="2026-07-09 18:59:02.357465932 +0000 UTC m=+42.808918393" Jul 09 18:59:02.488408 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.488379 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:02.567914 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.567884 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:02.693258 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.693221 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:59:02.693399 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.693275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:59:02.693399 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.693310 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:59:02.693399 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693380 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:59:02.693542 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693403 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:59:02.693542 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693419 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:59:02.693542 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693424 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:59:02.693542 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693456 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:59:10.693441127 +0000 UTC m=+51.144893566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:59:02.693542 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693476 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:10.693464768 +0000 UTC m=+51.144917207 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:59:02.693542 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:02.693491 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:10.69348371 +0000 UTC m=+51.144936148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:59:02.790921 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.790832 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:02.863160 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:02.863123 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:10.756286 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:10.756246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:10.756311 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:10.756347 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756390 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756428 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756459 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:26.756439951 +0000 UTC m=+67.207892390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756476 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:26.756465427 +0000 UTC m=+67.207917865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756474 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756497 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:59:10.756774 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:10.756577 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:59:26.756562712 +0000 UTC m=+67.208015170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:59:12.464660 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:12.464624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:12.564874 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:12.564836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:12.665611 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:12.665571 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:14.450459 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:14.450424 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:14.549830 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:14.549802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:18.369708 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:18.369673 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:21.302620 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:21.302585 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5s" Jul 09 18:59:21.451132 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:21.451093 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:21.504781 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:21.504747 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:25.183941 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.183902 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:25.238773 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.238742 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:25.764065 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.764030 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:59:25.766880 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.766861 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:59:25.778211 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.778181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/52c8d752-0a4d-4e02-85f9-435185c3da3d-original-pull-secret\") pod \"global-pull-secret-syncer-jfmg2\" (UID: \"52c8d752-0a4d-4e02-85f9-435185c3da3d\") " pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:59:25.815457 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.815425 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jfmg2" Jul 09 18:59:25.865259 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.864571 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 18:59:25.865259 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.864651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:59:25.867869 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.867371 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:59:25.867869 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.867696 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:59:25.875603 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:25.875233 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 18:59:25.875603 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:25.875312 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:29.875286524 +0000 UTC m=+130.326738962 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : secret "metrics-daemon-secret" not found Jul 09 18:59:25.877907 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.877700 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:59:25.898118 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.898073 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqbb\" (UniqueName: \"kubernetes.io/projected/a83ba2d3-877c-47dd-a2f4-b85ae700a975-kube-api-access-kxqbb\") pod \"network-check-target-89bcl\" (UID: \"a83ba2d3-877c-47dd-a2f4-b85ae700a975\") " pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:59:25.954468 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:25.954435 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jfmg2"] Jul 09 18:59:25.957331 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:59:25.957307 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c8d752_0a4d_4e02_85f9_435185c3da3d.slice/crio-2ed8b1a3f2576f5d7cd13df165f6f7379c78598d7728cab2ceadbb8230f7744b WatchSource:0}: Error finding container 2ed8b1a3f2576f5d7cd13df165f6f7379c78598d7728cab2ceadbb8230f7744b: Status 404 returned error can't find the container with id 2ed8b1a3f2576f5d7cd13df165f6f7379c78598d7728cab2ceadbb8230f7744b Jul 09 18:59:26.111273 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.111178 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rzlbw\"" Jul 09 18:59:26.119374 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.119348 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:59:26.238688 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.238660 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-89bcl"] Jul 09 18:59:26.241831 ip-10-0-133-17 kubenswrapper[2561]: W0709 18:59:26.241804 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda83ba2d3_877c_47dd_a2f4_b85ae700a975.slice/crio-7440d577bfd2da777931011bfc04740c267316ee5db95dd771a7a16b19ad400b WatchSource:0}: Error finding container 7440d577bfd2da777931011bfc04740c267316ee5db95dd771a7a16b19ad400b: Status 404 returned error can't find the container with id 7440d577bfd2da777931011bfc04740c267316ee5db95dd771a7a16b19ad400b Jul 09 18:59:26.385672 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.385584 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-89bcl" event={"ID":"a83ba2d3-877c-47dd-a2f4-b85ae700a975","Type":"ContainerStarted","Data":"7440d577bfd2da777931011bfc04740c267316ee5db95dd771a7a16b19ad400b"} Jul 09 18:59:26.386615 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.386593 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jfmg2" event={"ID":"52c8d752-0a4d-4e02-85f9-435185c3da3d","Type":"ContainerStarted","Data":"2ed8b1a3f2576f5d7cd13df165f6f7379c78598d7728cab2ceadbb8230f7744b"} Jul 09 18:59:26.772078 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.772033 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:59:26.772261 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.772120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:59:26.772261 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:26.772174 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:59:26.772261 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772198 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:59:26.772261 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772221 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:59:26.772451 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772282 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.77226364 +0000 UTC m=+99.223716095 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 18:59:26.772451 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772291 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:59:26.772451 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772291 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:59:26.772451 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772339 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.772323907 +0000 UTC m=+99.223776351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:59:26.772451 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:26.772357 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.772345752 +0000 UTC m=+99.223798190 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:59:29.725882 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:29.725845 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:29.786820 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:29.786782 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:30.538881 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:30.538842 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:30.597922 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:30.597888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:31.323942 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:31.323853 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:31.402239 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:31.402204 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-89bcl" event={"ID":"a83ba2d3-877c-47dd-a2f4-b85ae700a975","Type":"ContainerStarted","Data":"609896675c33a59bf34cc058a159b0f672ab8aaf4b8ab7bef88627b8420cd8d1"} Jul 09 18:59:31.402421 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:31.402304 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 18:59:31.403608 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:31.403586 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jfmg2" event={"ID":"52c8d752-0a4d-4e02-85f9-435185c3da3d","Type":"ContainerStarted","Data":"bfc5ed6dc04decf83b581688e5236328a758a7d700207d2f4f461b497dbc0554"} Jul 09 18:59:31.422706 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:31.420181 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-89bcl" podStartSLOduration=66.600279119 podStartE2EDuration="1m11.420163337s" totalImagesPullingTime="4.819884218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:31.419075973 +0000 UTC m=+71.870528434" watchObservedRunningTime="2026-07-09 18:59:31.420163337 +0000 UTC m=+71.871615798" Jul 09 18:59:31.436436 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:31.436393 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jfmg2" podStartSLOduration=66.328331132 podStartE2EDuration="1m11.436378526s" totalImagesPullingTime="5.108047394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:31.436064753 +0000 UTC m=+71.887517213" watchObservedRunningTime="2026-07-09 18:59:31.436378526 +0000 UTC m=+71.887831022" Jul 09 18:59:52.496371 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:52.496253 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:52.597398 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:52.597364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:58.794737 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:58.794692 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:58.794755 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: I0709 18:59:58.794806 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.794872 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.794892 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.794942 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.794968 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.794956 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 19:01:02.794942119 +0000 UTC m=+163.246394557 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.795052 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 19:01:02.795033299 +0000 UTC m=+163.246485752 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 18:59:58.795203 ip-10-0-133-17 kubenswrapper[2561]: E0709 18:59:58.795068 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 19:01:02.795059278 +0000 UTC m=+163.246511715 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 19:00:02.408653 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:02.408618 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-89bcl" Jul 09 19:00:22.418626 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:22.418595 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:22.517569 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:22.517535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:29.244390 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:29.244359 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:29.930155 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:29.930116 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:00:29.930327 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:00:29.930269 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 19:00:29.930367 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:00:29.930336 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:31.930318591 +0000 UTC m=+252.381771029 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : secret "metrics-daemon-secret" not found Jul 09 19:00:57.932283 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:00:57.932224 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" podUID="ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67" Jul 09 19:00:57.939674 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:00:57.939641 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" Jul 09 19:00:57.948809 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:00:57.948782 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mc95l" podUID="8f49887e-0428-4857-8f47-72d48641cea6" Jul 09 19:00:58.610476 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:58.610443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:00:58.610668 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:58.610442 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mc95l" Jul 09 19:00:58.610668 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:00:58.610442 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 19:00:59.120235 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:00:59.120175 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 19:01:02.869615 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:01:02.869561 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:01:02.869615 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:01:02.869629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:01:02.869662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869735 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869748 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869759 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f55989c55-9k6sd: secret "image-registry-tls" not found Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869772 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869825 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls podName:8f49887e-0428-4857-8f47-72d48641cea6 nodeName:}" failed. No retries permitted until 2026-07-09 19:03:04.869811745 +0000 UTC m=+285.321264183 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls") pod "dns-default-mc95l" (UID: "8f49887e-0428-4857-8f47-72d48641cea6") : secret "dns-default-metrics-tls" not found Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869839 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert podName:ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67 nodeName:}" failed. No retries permitted until 2026-07-09 19:03:04.869832822 +0000 UTC m=+285.321285259 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-btrnl" (UID: "ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67") : secret "networking-console-plugin-cert" not found Jul 09 19:01:02.870049 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:01:02.869849 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls podName:a9442778-7848-4690-a25a-882cf2c57aaf nodeName:}" failed. No retries permitted until 2026-07-09 19:03:04.869843461 +0000 UTC m=+285.321295899 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls") pod "image-registry-5f55989c55-9k6sd" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf") : secret "image-registry-tls" not found Jul 09 19:01:11.095272 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:01:11.095183 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:02:18.334574 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:18.334536 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:31.988794 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:31.988739 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:02:31.989198 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:31.988891 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 19:02:31.989198 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:31.988968 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs podName:3f59493d-083f-4990-8df2-09f3c8845e86 nodeName:}" failed. No retries permitted until 2026-07-09 19:04:33.988953086 +0000 UTC m=+374.440405524 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs") pod "network-metrics-daemon-stgpg" (UID: "3f59493d-083f-4990-8df2-09f3c8845e86") : secret "metrics-daemon-secret" not found Jul 09 19:02:39.907133 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.907104 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj"] Jul 09 19:02:39.908978 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.908962 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:39.911666 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.911647 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 19:02:39.911761 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.911726 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 19:02:39.912878 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.912859 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-szb2j\"" Jul 09 19:02:39.912967 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.912868 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 19:02:39.913120 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.912868 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 19:02:39.914604 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.914584 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz"] Jul 09 19:02:39.916321 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.916305 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z"] Jul 09 19:02:39.916434 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.916420 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:39.917866 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.917849 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c45df5c76-pqvtd"] Jul 09 19:02:39.917975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.917961 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:39.919073 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.919057 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9c6sd\"" Jul 09 19:02:39.919577 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.919560 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:39.920089 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920074 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 19:02:39.920382 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920362 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 19:02:39.920675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920659 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 19:02:39.920761 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920737 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 19:02:39.920822 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920760 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 19:02:39.920822 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920771 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q7gqf\"" Jul 09 19:02:39.920822 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920805 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 19:02:39.920987 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.920886 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 19:02:39.921119 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.921107 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 19:02:39.923665 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.923648 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 19:02:39.923907 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.923831 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 19:02:39.924008 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.923926 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 19:02:39.924008 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.923926 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 19:02:39.924159 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.924138 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 19:02:39.924251 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.924163 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5fd4t\"" Jul 09 19:02:39.924251 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.924162 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 19:02:39.927305 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.927284 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj"] Jul 09 19:02:39.933052 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.933029 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z"] Jul 09 19:02:39.938533 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.938494 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c45df5c76-pqvtd"] Jul 09 19:02:39.939344 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:39.939325 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz"] Jul 09 19:02:40.047192 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/306d86b2-299c-46f3-b01d-caf74eb20c23-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.047364 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047211 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-default-certificate\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.047364 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/839a1ce0-a0bd-4a90-873d-91be28e94261-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.047364 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047305 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clm4n\" (UniqueName: \"kubernetes.io/projected/839a1ce0-a0bd-4a90-873d-91be28e94261-kube-api-access-clm4n\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.047364 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-stats-auth\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.047575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.047575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047404 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d48a281-0903-416c-b93b-74557d5cb0cc-serving-cert\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.047575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d48a281-0903-416c-b93b-74557d5cb0cc-config\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.047575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047455 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.047575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047474 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlls\" (UniqueName: \"kubernetes.io/projected/1b789612-d416-4ae7-880e-71c9bc9939e3-kube-api-access-jtlls\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.047575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047535 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/839a1ce0-a0bd-4a90-873d-91be28e94261-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.047776 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcgcb\" (UniqueName: \"kubernetes.io/projected/5d48a281-0903-416c-b93b-74557d5cb0cc-kube-api-access-gcgcb\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.047776 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047625 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.047776 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.047643 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5mhr\" (UniqueName: \"kubernetes.io/projected/306d86b2-299c-46f3-b01d-caf74eb20c23-kube-api-access-k5mhr\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.148200 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d48a281-0903-416c-b93b-74557d5cb0cc-serving-cert\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.148200 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148197 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d48a281-0903-416c-b93b-74557d5cb0cc-config\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148221 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlls\" (UniqueName: \"kubernetes.io/projected/1b789612-d416-4ae7-880e-71c9bc9939e3-kube-api-access-jtlls\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/839a1ce0-a0bd-4a90-873d-91be28e94261-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.148334 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148406 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcgcb\" (UniqueName: \"kubernetes.io/projected/5d48a281-0903-416c-b93b-74557d5cb0cc-kube-api-access-gcgcb\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.148415 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:40.648396351 +0000 UTC m=+261.099848809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : secret "router-metrics-certs-default" not found Jul 09 19:02:40.148469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148450 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.148912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148480 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5mhr\" (UniqueName: \"kubernetes.io/projected/306d86b2-299c-46f3-b01d-caf74eb20c23-kube-api-access-k5mhr\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.148912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148534 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/306d86b2-299c-46f3-b01d-caf74eb20c23-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.148912 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.148762 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:40.148912 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.148825 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls podName:306d86b2-299c-46f3-b01d-caf74eb20c23 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:40.648811396 +0000 UTC m=+261.100263836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cn5rz" (UID: "306d86b2-299c-46f3-b01d-caf74eb20c23") : secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:40.148912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-default-certificate\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.148912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/839a1ce0-a0bd-4a90-873d-91be28e94261-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.149395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.148932 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d48a281-0903-416c-b93b-74557d5cb0cc-config\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.149395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.149192 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clm4n\" (UniqueName: \"kubernetes.io/projected/839a1ce0-a0bd-4a90-873d-91be28e94261-kube-api-access-clm4n\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.149395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.149283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-stats-auth\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.149395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.149325 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.149395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.149336 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/306d86b2-299c-46f3-b01d-caf74eb20c23-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.149670 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.149473 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:40.649456761 +0000 UTC m=+261.100909255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : configmap references non-existent config key: service-ca.crt Jul 09 19:02:40.149670 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.149487 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/839a1ce0-a0bd-4a90-873d-91be28e94261-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.150922 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.150889 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d48a281-0903-416c-b93b-74557d5cb0cc-serving-cert\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.150922 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.150908 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/839a1ce0-a0bd-4a90-873d-91be28e94261-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.151293 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.151278 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-default-certificate\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.151684 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.151669 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-stats-auth\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.157973 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.157901 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clm4n\" (UniqueName: \"kubernetes.io/projected/839a1ce0-a0bd-4a90-873d-91be28e94261-kube-api-access-clm4n\") pod \"kube-storage-version-migrator-operator-799cb7f958-wjndj\" (UID: \"839a1ce0-a0bd-4a90-873d-91be28e94261\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.158120 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.158060 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcgcb\" (UniqueName: \"kubernetes.io/projected/5d48a281-0903-416c-b93b-74557d5cb0cc-kube-api-access-gcgcb\") pod \"service-ca-operator-7695fc6c8c-pjh5z\" (UID: \"5d48a281-0903-416c-b93b-74557d5cb0cc\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.158253 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.158227 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlls\" (UniqueName: \"kubernetes.io/projected/1b789612-d416-4ae7-880e-71c9bc9939e3-kube-api-access-jtlls\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.158676 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.158657 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5mhr\" (UniqueName: \"kubernetes.io/projected/306d86b2-299c-46f3-b01d-caf74eb20c23-kube-api-access-k5mhr\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.160310 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.160264 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:40.218218 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.218178 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" Jul 09 19:02:40.232958 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.232931 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" Jul 09 19:02:40.352251 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.352217 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj"] Jul 09 19:02:40.354869 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:02:40.354848 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839a1ce0_a0bd_4a90_873d_91be28e94261.slice/crio-eee6d722b52638205db193facb9dbbcea5e23196f663f51e56828b347b645f58 WatchSource:0}: Error finding container eee6d722b52638205db193facb9dbbcea5e23196f663f51e56828b347b645f58: Status 404 returned error can't find the container with id eee6d722b52638205db193facb9dbbcea5e23196f663f51e56828b347b645f58 Jul 09 19:02:40.365749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.365725 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z"] Jul 09 19:02:40.368544 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:02:40.368494 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d48a281_0903_416c_b93b_74557d5cb0cc.slice/crio-abcf840562cb40d324f8df242c508271ad56f6c4b656016749a9e89eb2f7628a WatchSource:0}: Error finding container abcf840562cb40d324f8df242c508271ad56f6c4b656016749a9e89eb2f7628a: Status 404 returned error can't find the container with id abcf840562cb40d324f8df242c508271ad56f6c4b656016749a9e89eb2f7628a Jul 09 19:02:40.653722 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.653667 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:40.653907 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.653785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.653907 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.653840 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:40.653907 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.653904 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 19:02:40.654001 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.653911 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls podName:306d86b2-299c-46f3-b01d-caf74eb20c23 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:41.653890891 +0000 UTC m=+262.105343328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cn5rz" (UID: "306d86b2-299c-46f3-b01d-caf74eb20c23") : secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:40.654001 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.653844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:40.654001 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.653936 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:41.65392635 +0000 UTC m=+262.105378788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : secret "router-metrics-certs-default" not found Jul 09 19:02:40.654001 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:40.653973 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:41.653949097 +0000 UTC m=+262.105401538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : configmap references non-existent config key: service-ca.crt Jul 09 19:02:40.852973 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.852893 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" event={"ID":"5d48a281-0903-416c-b93b-74557d5cb0cc","Type":"ContainerStarted","Data":"abcf840562cb40d324f8df242c508271ad56f6c4b656016749a9e89eb2f7628a"} Jul 09 19:02:40.854008 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:40.853987 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" event={"ID":"839a1ce0-a0bd-4a90-873d-91be28e94261","Type":"ContainerStarted","Data":"eee6d722b52638205db193facb9dbbcea5e23196f663f51e56828b347b645f58"} Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:41.663135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:41.663213 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:41.663257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:41.663349 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:43.66333048 +0000 UTC m=+264.114782927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : configmap references non-existent config key: service-ca.crt Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:41.663435 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:41.663472 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:43.663461226 +0000 UTC m=+264.114913668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : secret "router-metrics-certs-default" not found Jul 09 19:02:41.663539 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:41.663552 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:41.664294 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:41.663592 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls podName:306d86b2-299c-46f3-b01d-caf74eb20c23 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:43.663580721 +0000 UTC m=+264.115033159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cn5rz" (UID: "306d86b2-299c-46f3-b01d-caf74eb20c23") : secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:41.955695 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:41.955608 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:43.677382 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.677283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.677401 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:43.677435 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.677458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:43.677528 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls podName:306d86b2-299c-46f3-b01d-caf74eb20c23 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:47.677488335 +0000 UTC m=+268.128940780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cn5rz" (UID: "306d86b2-299c-46f3-b01d-caf74eb20c23") : secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:43.677563 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:47.677552017 +0000 UTC m=+268.129004456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : configmap references non-existent config key: service-ca.crt Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:43.677574 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 19:02:43.677782 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:43.677624 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:47.677609406 +0000 UTC m=+268.129061851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : secret "router-metrics-certs-default" not found Jul 09 19:02:43.865803 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.865767 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" event={"ID":"5d48a281-0903-416c-b93b-74557d5cb0cc","Type":"ContainerStarted","Data":"7e6d7d7bed16dc54f4f7ba7bddf78386484314cbf379ca05f7b1f89c5e331154"} Jul 09 19:02:43.867203 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.867178 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" event={"ID":"839a1ce0-a0bd-4a90-873d-91be28e94261","Type":"ContainerStarted","Data":"eeb8a85f5720b3a9acc2a6c3c7ce393b3a177c1dd1e2dd1ce43e134670c1b757"} Jul 09 19:02:43.882609 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.882556 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" podStartSLOduration=1.844532058 podStartE2EDuration="4.88253882s" totalImagesPullingTime="3.038006762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:02:43.881918254 +0000 UTC m=+264.333370711" watchObservedRunningTime="2026-07-09 19:02:43.88253882 +0000 UTC m=+264.333991283" Jul 09 19:02:43.898118 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:43.898058 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" podStartSLOduration=1.845620968 podStartE2EDuration="4.898036111s" totalImagesPullingTime="3.052415143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:02:43.896783123 +0000 UTC m=+264.348235583" watchObservedRunningTime="2026-07-09 19:02:43.898036111 +0000 UTC m=+264.349488573" Jul 09 19:02:44.007921 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.007887 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:44.058854 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.058796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:44.294001 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.293921 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-58bnj"] Jul 09 19:02:44.296129 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.296113 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.298962 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.298943 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 19:02:44.299091 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.298998 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 19:02:44.300103 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.300082 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 19:02:44.300204 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.300131 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 19:02:44.300204 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.300084 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rw89p\"" Jul 09 19:02:44.316808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.316784 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-58bnj"] Jul 09 19:02:44.338235 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.338206 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:44.384478 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.384433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.384668 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.384490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf9z4\" (UniqueName: \"kubernetes.io/projected/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-api-access-zf9z4\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.384668 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.384596 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f57b8ec-116b-428c-a6a9-36300b01abb3-data-volume\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.384786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.384661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f57b8ec-116b-428c-a6a9-36300b01abb3-crio-socket\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.384786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.384777 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.398760 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.398702 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:44.485254 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485220 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f57b8ec-116b-428c-a6a9-36300b01abb3-data-volume\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.485441 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f57b8ec-116b-428c-a6a9-36300b01abb3-crio-socket\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.485441 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485368 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.485441 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485425 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.485608 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485447 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f57b8ec-116b-428c-a6a9-36300b01abb3-crio-socket\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.485608 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf9z4\" (UniqueName: \"kubernetes.io/projected/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-api-access-zf9z4\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.485608 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.485545 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 19:02:44.485608 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.485569 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 19:02:44.485744 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.485611 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:44.985591532 +0000 UTC m=+265.437043969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : configmap "kube-rbac-proxy" not found Jul 09 19:02:44.485744 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.485629 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:44.985619589 +0000 UTC m=+265.437072029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : secret "insights-runtime-extractor-tls" not found Jul 09 19:02:44.485744 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.485655 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f57b8ec-116b-428c-a6a9-36300b01abb3-data-volume\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.494326 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.494296 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf9z4\" (UniqueName: \"kubernetes.io/projected/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-api-access-zf9z4\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.718759 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.718667 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:44.820047 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.820015 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:44.988933 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.988896 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.989097 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.989073 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 19:02:44.989097 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:44.989085 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:44.989210 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.989148 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:45.989126053 +0000 UTC m=+266.440578496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : secret "insights-runtime-extractor-tls" not found Jul 09 19:02:44.989210 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.989166 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 19:02:44.989210 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:44.989206 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:45.989195352 +0000 UTC m=+266.440647791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : configmap "kube-rbac-proxy" not found Jul 09 19:02:45.180723 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:45.180692 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:45.271127 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:45.271041 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:45.372418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:45.372376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:45.472902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:45.472839 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:45.997013 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:45.996977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:45.997427 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:45.997036 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:45.997427 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:45.997100 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 19:02:45.997427 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:45.997180 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:47.997163653 +0000 UTC m=+268.448616090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : configmap "kube-rbac-proxy" not found Jul 09 19:02:45.997427 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:45.997220 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 19:02:45.997427 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:45.997298 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:47.997280194 +0000 UTC m=+268.448732647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : secret "insights-runtime-extractor-tls" not found Jul 09 19:02:46.377546 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:46.377441 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rk9t5_56b19497-c17c-4c4e-85fc-9e1ceedaa60f/dns-node-resolver/0.log" Jul 09 19:02:47.022987 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.022951 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-5ljds"] Jul 09 19:02:47.025279 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.025256 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.028112 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.028091 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 19:02:47.028239 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.028096 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 19:02:47.029429 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.029407 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pffkh\"" Jul 09 19:02:47.029554 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.029449 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 19:02:47.029554 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.029457 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 19:02:47.036387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.036366 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-5ljds"] Jul 09 19:02:47.061018 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.060991 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:47.106323 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.106289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmhsc\" (UniqueName: \"kubernetes.io/projected/ee374798-f2fe-454c-9a9f-fe611d75a2b4-kube-api-access-mmhsc\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.106452 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.106391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee374798-f2fe-454c-9a9f-fe611d75a2b4-signing-cabundle\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.106452 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.106435 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee374798-f2fe-454c-9a9f-fe611d75a2b4-signing-key\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.133718 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.133690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:47.182362 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.182336 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2mbf8_080589b7-abe1-4464-a9eb-37eb06937242/node-ca/0.log" Jul 09 19:02:47.207365 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.207334 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmhsc\" (UniqueName: \"kubernetes.io/projected/ee374798-f2fe-454c-9a9f-fe611d75a2b4-kube-api-access-mmhsc\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.207493 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.207395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee374798-f2fe-454c-9a9f-fe611d75a2b4-signing-cabundle\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.207493 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.207436 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee374798-f2fe-454c-9a9f-fe611d75a2b4-signing-key\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.208182 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.208165 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee374798-f2fe-454c-9a9f-fe611d75a2b4-signing-cabundle\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.209932 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.209915 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee374798-f2fe-454c-9a9f-fe611d75a2b4-signing-key\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.215913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.215894 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmhsc\" (UniqueName: \"kubernetes.io/projected/ee374798-f2fe-454c-9a9f-fe611d75a2b4-kube-api-access-mmhsc\") pod \"service-ca-79f68b9f4f-5ljds\" (UID: \"ee374798-f2fe-454c-9a9f-fe611d75a2b4\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.334195 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.334125 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" Jul 09 19:02:47.455172 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.455139 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-5ljds"] Jul 09 19:02:47.457867 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:02:47.457844 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee374798_f2fe_454c_9a9f_fe611d75a2b4.slice/crio-4a491f3c0c3f1402a3a05a25f34d4c0fd07e99874af144102cd6b91378b861ad WatchSource:0}: Error finding container 4a491f3c0c3f1402a3a05a25f34d4c0fd07e99874af144102cd6b91378b861ad: Status 404 returned error can't find the container with id 4a491f3c0c3f1402a3a05a25f34d4c0fd07e99874af144102cd6b91378b861ad Jul 09 19:02:47.710702 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.710605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:47.710702 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.710692 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:47.710937 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.710741 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:47.710937 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:47.710770 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:55.71075212 +0000 UTC m=+276.162204558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : configmap references non-existent config key: service-ca.crt Jul 09 19:02:47.710937 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:47.710831 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 19:02:47.710937 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:47.710910 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs podName:1b789612-d416-4ae7-880e-71c9bc9939e3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:55.710898157 +0000 UTC m=+276.162350595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs") pod "router-default-7c45df5c76-pqvtd" (UID: "1b789612-d416-4ae7-880e-71c9bc9939e3") : secret "router-metrics-certs-default" not found Jul 09 19:02:47.710937 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:47.710833 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:47.711180 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:47.710958 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls podName:306d86b2-299c-46f3-b01d-caf74eb20c23 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:55.71094769 +0000 UTC m=+276.162400129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cn5rz" (UID: "306d86b2-299c-46f3-b01d-caf74eb20c23") : secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:47.884225 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.884182 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" event={"ID":"ee374798-f2fe-454c-9a9f-fe611d75a2b4","Type":"ContainerStarted","Data":"92359705d7b4f3764f5d1e271551e9681c695746a860d36e507c9ae858398b31"} Jul 09 19:02:47.884399 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.884244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" event={"ID":"ee374798-f2fe-454c-9a9f-fe611d75a2b4","Type":"ContainerStarted","Data":"4a491f3c0c3f1402a3a05a25f34d4c0fd07e99874af144102cd6b91378b861ad"} Jul 09 19:02:47.901402 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:47.901343 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-5ljds" podStartSLOduration=0.901324884 podStartE2EDuration="901.324884ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:02:47.901276097 +0000 UTC m=+268.352728559" watchObservedRunningTime="2026-07-09 19:02:47.901324884 +0000 UTC m=+268.352777343" Jul 09 19:02:48.012377 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.012344 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:48.012558 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.012403 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:48.012558 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:48.012479 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 19:02:48.012558 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:48.012522 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 19:02:48.012657 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:48.012561 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:52.012545701 +0000 UTC m=+272.463998138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : configmap "kube-rbac-proxy" not found Jul 09 19:02:48.012657 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:48.012576 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:02:52.012570527 +0000 UTC m=+272.464022966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : secret "insights-runtime-extractor-tls" not found Jul 09 19:02:48.116686 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.116634 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:48.176592 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.176560 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dkchf_6736da78-2995-4f4b-b579-543e73b3fbd3/serve-healthcheck-canary/0.log" Jul 09 19:02:48.176878 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.176849 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:48.372841 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.372753 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:48.800395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.800309 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:48.972137 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:48.972101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:49.115886 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:49.115784 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:49.219851 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:49.219809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:49.315932 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:49.315901 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:49.418784 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:49.418684 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:49.649778 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:49.649743 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:49.848742 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:49.848709 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:51.042263 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:51.042221 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:51.244854 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:51.244806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:51.428905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:51.428812 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:51.485813 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:51.485778 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:51.653905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:51.653871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:51.846813 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:51.846776 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:52.050737 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:52.050700 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:52.051105 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:52.050783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:02:52.051105 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:52.050818 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 19:02:52.051105 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:52.050884 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:03:00.050868091 +0000 UTC m=+280.502320530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : configmap "kube-rbac-proxy" not found Jul 09 19:02:52.051105 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:52.050930 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 19:02:52.051105 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:52.050985 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls podName:6f57b8ec-116b-428c-a6a9-36300b01abb3 nodeName:}" failed. No retries permitted until 2026-07-09 19:03:00.050972171 +0000 UTC m=+280.502424610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58bnj" (UID: "6f57b8ec-116b-428c-a6a9-36300b01abb3") : secret "insights-runtime-extractor-tls" not found Jul 09 19:02:52.644681 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:52.644644 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:52.701118 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:52.701083 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:52.843117 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:52.843087 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:53.053089 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:53.053052 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:53.446585 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:53.446480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:53.660749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:53.660698 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:53.895643 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:53.895602 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:54.247258 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:54.247228 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:54.443447 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:54.443421 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:55.123997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.123956 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:55.225935 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.225904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:55.324895 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.324860 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:55.447282 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.447187 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:55.645827 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.645791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:55.783154 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.783120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:55.783347 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.783172 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:02:55.783347 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.783245 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:55.783347 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:55.783319 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:55.783535 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:02:55.783377 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls podName:306d86b2-299c-46f3-b01d-caf74eb20c23 nodeName:}" failed. No retries permitted until 2026-07-09 19:03:11.783362119 +0000 UTC m=+292.234814557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-cn5rz" (UID: "306d86b2-299c-46f3-b01d-caf74eb20c23") : secret "cluster-monitoring-operator-tls" not found Jul 09 19:02:55.783837 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.783819 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b789612-d416-4ae7-880e-71c9bc9939e3-service-ca-bundle\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:55.785661 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.785643 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b789612-d416-4ae7-880e-71c9bc9939e3-metrics-certs\") pod \"router-default-7c45df5c76-pqvtd\" (UID: \"1b789612-d416-4ae7-880e-71c9bc9939e3\") " pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:55.839912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.839882 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:55.986058 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:55.986029 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c45df5c76-pqvtd"] Jul 09 19:02:55.986930 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:02:55.986903 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b789612_d416_4ae7_880e_71c9bc9939e3.slice/crio-52f695e4d6c699a7840790f07ac16f19cb318afe51d0007f80d27bd61c4fa365 WatchSource:0}: Error finding container 52f695e4d6c699a7840790f07ac16f19cb318afe51d0007f80d27bd61c4fa365: Status 404 returned error can't find the container with id 52f695e4d6c699a7840790f07ac16f19cb318afe51d0007f80d27bd61c4fa365 Jul 09 19:02:56.046130 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:56.046096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:56.646615 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:56.646584 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:56.846391 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:56.846357 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:56.912156 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:56.912077 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" event={"ID":"1b789612-d416-4ae7-880e-71c9bc9939e3","Type":"ContainerStarted","Data":"6b4145ee1db0fe5448a1b9005d63df291c0d417fdd375a3f6f5c4bb6e82d2537"} Jul 09 19:02:56.912156 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:56.912123 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" event={"ID":"1b789612-d416-4ae7-880e-71c9bc9939e3","Type":"ContainerStarted","Data":"52f695e4d6c699a7840790f07ac16f19cb318afe51d0007f80d27bd61c4fa365"} Jul 09 19:02:56.930158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:56.930097 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" podStartSLOduration=17.930077743 podStartE2EDuration="17.930077743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:02:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:02:56.929766298 +0000 UTC m=+277.381218757" watchObservedRunningTime="2026-07-09 19:02:56.930077743 +0000 UTC m=+277.381530196" Jul 09 19:02:57.449077 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.449039 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:57.600176 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.600143 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:57.840865 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.840824 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:57.843586 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.843563 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:57.849002 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.848970 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:57.915980 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.915951 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:57.917213 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:57.917187 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c45df5c76-pqvtd" Jul 09 19:02:58.042139 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:58.042099 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:58.446169 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:58.446135 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:59.046877 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:59.046829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:59.689392 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:59.689353 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:02:59.744394 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:02:59.744363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:00.121045 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.121004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:03:00.121407 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.121111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:03:00.121656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.121635 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f57b8ec-116b-428c-a6a9-36300b01abb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:03:00.123421 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.123398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f57b8ec-116b-428c-a6a9-36300b01abb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58bnj\" (UID: \"6f57b8ec-116b-428c-a6a9-36300b01abb3\") " pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:03:00.206915 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.206876 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-58bnj" Jul 09 19:03:00.337344 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.337311 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-58bnj"] Jul 09 19:03:00.340279 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:00.340254 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f57b8ec_116b_428c_a6a9_36300b01abb3.slice/crio-b992a0df09ae9c8e3725701d1639123fd6d4542c70e8a4c437373f59f6a06050 WatchSource:0}: Error finding container b992a0df09ae9c8e3725701d1639123fd6d4542c70e8a4c437373f59f6a06050: Status 404 returned error can't find the container with id b992a0df09ae9c8e3725701d1639123fd6d4542c70e8a4c437373f59f6a06050 Jul 09 19:03:00.929302 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:00.929268 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58bnj" event={"ID":"6f57b8ec-116b-428c-a6a9-36300b01abb3","Type":"ContainerStarted","Data":"b992a0df09ae9c8e3725701d1639123fd6d4542c70e8a4c437373f59f6a06050"} Jul 09 19:03:01.611266 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:01.611191 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" podUID="ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67" Jul 09 19:03:01.611266 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:01.611190 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" Jul 09 19:03:01.611266 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:01.611200 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mc95l" podUID="8f49887e-0428-4857-8f47-72d48641cea6" Jul 09 19:03:01.842494 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:01.842432 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:01.932750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:01.932670 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:01.932927 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:01.932669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 19:03:01.932927 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:01.932670 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mc95l" Jul 09 19:03:01.942258 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:01.942226 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:01.998434 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:01.998366 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:02.160684 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:02.160615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:02.663895 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:02.663842 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:02.765412 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:02.765369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:02.922247 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:02.922099 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:02.984325 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:02.984237 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:03.370436 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:03.370375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:03.428126 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:03.428084 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:03.482732 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:03.482696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:03.756374 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:03.756324 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:03.948391 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:03.948355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:04.621296 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.621197 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:04.756166 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.756128 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:04.943589 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.943558 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58bnj" event={"ID":"6f57b8ec-116b-428c-a6a9-36300b01abb3","Type":"ContainerStarted","Data":"03f0221ef07fa12606418b544882049c8eec32ee12b0b8548d80da460cec9e61"} Jul 09 19:03:04.943929 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.943620 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58bnj" event={"ID":"6f57b8ec-116b-428c-a6a9-36300b01abb3","Type":"ContainerStarted","Data":"8a046ebb5fc04511640c9cef6d185741600ec1938f8218662bd9e0a03d8c47ab"} Jul 09 19:03:04.968876 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.968847 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 19:03:04.969000 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.968892 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:04.969000 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.968937 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 19:03:04.971525 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.971484 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-btrnl\" (UID: \"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 19:03:04.971631 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.971496 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f49887e-0428-4857-8f47-72d48641cea6-metrics-tls\") pod \"dns-default-mc95l\" (UID: \"8f49887e-0428-4857-8f47-72d48641cea6\") " pod="openshift-dns/dns-default-mc95l" Jul 09 19:03:04.971631 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:04.971598 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"image-registry-5f55989c55-9k6sd\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:05.238138 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.238109 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bw6lk\"" Jul 09 19:03:05.238138 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.238127 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xh4zm\"" Jul 09 19:03:05.238355 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.238109 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9x2bv\"" Jul 09 19:03:05.244390 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.244365 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mc95l" Jul 09 19:03:05.244544 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.244444 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" Jul 09 19:03:05.244544 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.244474 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:05.443165 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.443120 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f55989c55-9k6sd"] Jul 09 19:03:05.447501 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:05.447466 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9442778_7848_4690_a25a_882cf2c57aaf.slice/crio-1a459c484e79156a1f68af378fba421aef7a8d953d776157be33002fda93a000 WatchSource:0}: Error finding container 1a459c484e79156a1f68af378fba421aef7a8d953d776157be33002fda93a000: Status 404 returned error can't find the container with id 1a459c484e79156a1f68af378fba421aef7a8d953d776157be33002fda93a000 Jul 09 19:03:05.613958 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.613927 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-btrnl"] Jul 09 19:03:05.617657 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:05.617620 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7b42d7_c9a1_4f9c_96ec_53f1b1567a67.slice/crio-c81a993e44c9987587496952a56ff81bafc0296aed05f3e9926d2ae63ec0c121 WatchSource:0}: Error finding container c81a993e44c9987587496952a56ff81bafc0296aed05f3e9926d2ae63ec0c121: Status 404 returned error can't find the container with id c81a993e44c9987587496952a56ff81bafc0296aed05f3e9926d2ae63ec0c121 Jul 09 19:03:05.618805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.618780 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mc95l"] Jul 09 19:03:05.622267 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:05.622245 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f49887e_0428_4857_8f47_72d48641cea6.slice/crio-02c65f439e521179495f779568a248996e5882a7bf20433deccc482b971f2dc3 WatchSource:0}: Error finding container 02c65f439e521179495f779568a248996e5882a7bf20433deccc482b971f2dc3: Status 404 returned error can't find the container with id 02c65f439e521179495f779568a248996e5882a7bf20433deccc482b971f2dc3 Jul 09 19:03:05.948731 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.948637 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" event={"ID":"a9442778-7848-4690-a25a-882cf2c57aaf","Type":"ContainerStarted","Data":"fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669"} Jul 09 19:03:05.948731 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.948696 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" event={"ID":"a9442778-7848-4690-a25a-882cf2c57aaf","Type":"ContainerStarted","Data":"1a459c484e79156a1f68af378fba421aef7a8d953d776157be33002fda93a000"} Jul 09 19:03:05.948731 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.948726 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:05.950019 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.949990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" event={"ID":"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67","Type":"ContainerStarted","Data":"c81a993e44c9987587496952a56ff81bafc0296aed05f3e9926d2ae63ec0c121"} Jul 09 19:03:05.951166 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.951132 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mc95l" event={"ID":"8f49887e-0428-4857-8f47-72d48641cea6","Type":"ContainerStarted","Data":"02c65f439e521179495f779568a248996e5882a7bf20433deccc482b971f2dc3"} Jul 09 19:03:05.970333 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:05.970265 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" podStartSLOduration=285.970241384 podStartE2EDuration="4m45.970241384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:05.96843704 +0000 UTC m=+286.419889517" watchObservedRunningTime="2026-07-09 19:03:05.970241384 +0000 UTC m=+286.421693845" Jul 09 19:03:07.961496 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:07.961459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58bnj" event={"ID":"6f57b8ec-116b-428c-a6a9-36300b01abb3","Type":"ContainerStarted","Data":"7fc7bb4f8f198366cdfb22d04926268c4aa280dc1adfdf4ed32201438d5eefa2"} Jul 09 19:03:07.963295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:07.963257 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" event={"ID":"ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67","Type":"ContainerStarted","Data":"f0c4d5b9b8c24d4ed8981aa8c4fe9d42dd7bc1d5d2824a93a47bb165a01be63c"} Jul 09 19:03:07.980625 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:07.980570 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-58bnj" podStartSLOduration=16.704753312 podStartE2EDuration="23.980548284s" totalImagesPullingTime="7.275794972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:02:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:07.978000331 +0000 UTC m=+288.429452791" watchObservedRunningTime="2026-07-09 19:03:07.980548284 +0000 UTC m=+288.432000742" Jul 09 19:03:07.994290 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:07.994231 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-btrnl" podStartSLOduration=262.887284958 podStartE2EDuration="4m24.994209937s" totalImagesPullingTime="2.106924979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:07.993906577 +0000 UTC m=+288.445359038" watchObservedRunningTime="2026-07-09 19:03:07.994209937 +0000 UTC m=+288.445662398" Jul 09 19:03:08.791492 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:08.791395 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:08.889808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:08.889714 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:08.967933 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:08.967895 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mc95l" event={"ID":"8f49887e-0428-4857-8f47-72d48641cea6","Type":"ContainerStarted","Data":"5af07035d8c5a30b5364fa77dc8451de074f1f85a3c1e81b8127316c3ab4b9d9"} Jul 09 19:03:08.968342 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:08.967962 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mc95l" event={"ID":"8f49887e-0428-4857-8f47-72d48641cea6","Type":"ContainerStarted","Data":"8a21b96c6d78f4c71ab7941eddbccd0dcd212a1fc3b73787122fba26843a2bb1"} Jul 09 19:03:08.986673 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:08.986624 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mc95l" podStartSLOduration=251.878915222 podStartE2EDuration="4m14.986606922s" totalImagesPullingTime="3.1076917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:08.986089365 +0000 UTC m=+289.437541824" watchObservedRunningTime="2026-07-09 19:03:08.986606922 +0000 UTC m=+289.438059382" Jul 09 19:03:08.988212 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:08.988169 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:09.129772 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:09.129739 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:09.218979 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:09.218912 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:09.971067 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:09.971029 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mc95l" Jul 09 19:03:11.830460 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.830409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:03:11.833082 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.833054 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/306d86b2-299c-46f3-b01d-caf74eb20c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-cn5rz\" (UID: \"306d86b2-299c-46f3-b01d-caf74eb20c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:03:11.959234 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.959195 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-597f649fcf-4kqzp"] Jul 09 19:03:11.962415 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.962393 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:11.965244 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.965200 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 19:03:11.965244 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.965220 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 19:03:11.965427 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.965273 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 19:03:11.966339 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.966316 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 19:03:11.966452 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.966332 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 19:03:11.966452 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.966400 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 19:03:11.966595 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.966449 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c5qfs\"" Jul 09 19:03:11.966595 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.966483 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 19:03:11.972972 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:11.972947 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597f649fcf-4kqzp"] Jul 09 19:03:12.027127 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.027083 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" Jul 09 19:03:12.032435 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.032416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-service-ca\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.032542 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.032443 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-oauth-serving-cert\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.032542 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.032467 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-oauth-config\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.032634 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.032566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-serving-cert\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.032634 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.032613 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/849cdd66-6726-41a0-a945-3804ee1fa45d-kube-api-access-l8tz4\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.032716 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.032652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-console-config\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134003 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-oauth-config\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134072 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-serving-cert\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/849cdd66-6726-41a0-a945-3804ee1fa45d-kube-api-access-l8tz4\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-console-config\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-service-ca\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-oauth-serving-cert\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.134942 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.134906 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-service-ca\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.135199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.135175 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-oauth-serving-cert\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.135249 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.135214 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-console-config\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.136732 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.136703 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-oauth-config\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.136824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.136770 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-serving-cert\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.142258 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.142206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/849cdd66-6726-41a0-a945-3804ee1fa45d-kube-api-access-l8tz4\") pod \"console-597f649fcf-4kqzp\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.156358 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.156328 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz"] Jul 09 19:03:12.159481 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:12.159454 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306d86b2_299c_46f3_b01d_caf74eb20c23.slice/crio-cfd2b6ff3b9de7d803b7f97f9fb8b1465ea8625da31ba7e85ee6eee34a141af4 WatchSource:0}: Error finding container cfd2b6ff3b9de7d803b7f97f9fb8b1465ea8625da31ba7e85ee6eee34a141af4: Status 404 returned error can't find the container with id cfd2b6ff3b9de7d803b7f97f9fb8b1465ea8625da31ba7e85ee6eee34a141af4 Jul 09 19:03:12.271458 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.271415 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:12.401717 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.401635 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597f649fcf-4kqzp"] Jul 09 19:03:12.405861 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:12.405834 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849cdd66_6726_41a0_a945_3804ee1fa45d.slice/crio-14c24ebeba78c0f52dc01dbf57654ac4f2c9927ef39b9d38b244ddd641a4d04c WatchSource:0}: Error finding container 14c24ebeba78c0f52dc01dbf57654ac4f2c9927ef39b9d38b244ddd641a4d04c: Status 404 returned error can't find the container with id 14c24ebeba78c0f52dc01dbf57654ac4f2c9927ef39b9d38b244ddd641a4d04c Jul 09 19:03:12.981484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.981447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597f649fcf-4kqzp" event={"ID":"849cdd66-6726-41a0-a945-3804ee1fa45d","Type":"ContainerStarted","Data":"14c24ebeba78c0f52dc01dbf57654ac4f2c9927ef39b9d38b244ddd641a4d04c"} Jul 09 19:03:12.982586 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:12.982566 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" event={"ID":"306d86b2-299c-46f3-b01d-caf74eb20c23","Type":"ContainerStarted","Data":"cfd2b6ff3b9de7d803b7f97f9fb8b1465ea8625da31ba7e85ee6eee34a141af4"} Jul 09 19:03:14.096019 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:14.095980 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-stgpg" podUID="3f59493d-083f-4990-8df2-09f3c8845e86" Jul 09 19:03:14.363250 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:14.363083 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:14.463750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:14.463675 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:14.533298 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:14.533262 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:14.639808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:14.639715 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:14.991829 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:14.991797 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" event={"ID":"306d86b2-299c-46f3-b01d-caf74eb20c23","Type":"ContainerStarted","Data":"f50077578bd8ae318c13bf766d7620aaa5dd6d03129a97d06dafed891a9a11b7"} Jul 09 19:03:15.009273 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:15.009221 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-cn5rz" podStartSLOduration=34.146944234 podStartE2EDuration="36.009202291s" totalImagesPullingTime="1.862258057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:15.00831501 +0000 UTC m=+295.459767470" watchObservedRunningTime="2026-07-09 19:03:15.009202291 +0000 UTC m=+295.460654751" Jul 09 19:03:15.759142 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:15.759087 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:15.815819 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:15.815779 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:16.951453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:16.951420 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:17.000399 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.000366 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597f649fcf-4kqzp" event={"ID":"849cdd66-6726-41a0-a945-3804ee1fa45d","Type":"ContainerStarted","Data":"596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097"} Jul 09 19:03:17.012283 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.012246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:17.019473 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.019423 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-597f649fcf-4kqzp" podStartSLOduration=2.351603436 podStartE2EDuration="6.019409172s" totalImagesPullingTime="3.667805736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:17.017995374 +0000 UTC m=+297.469447834" watchObservedRunningTime="2026-07-09 19:03:17.019409172 +0000 UTC m=+297.470861632" Jul 09 19:03:17.545004 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.544966 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:17.580847 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.580813 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wfphb"] Jul 09 19:03:17.583597 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.583581 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.586943 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.586916 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 19:03:17.588353 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.588336 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 19:03:17.588446 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.588377 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 19:03:17.588446 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.588380 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kl8qg\"" Jul 09 19:03:17.595723 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.595694 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wfphb"] Jul 09 19:03:17.617707 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.617660 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:17.683767 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.683725 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.683913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.683834 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-777kb\" (UniqueName: \"kubernetes.io/projected/d8a09f1e-f229-4f72-b717-838d5b300f1c-kube-api-access-777kb\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.683913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.683878 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8a09f1e-f229-4f72-b717-838d5b300f1c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.683913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.683902 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.784687 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.784649 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.784836 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.784709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-777kb\" (UniqueName: \"kubernetes.io/projected/d8a09f1e-f229-4f72-b717-838d5b300f1c-kube-api-access-777kb\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.784836 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.784730 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8a09f1e-f229-4f72-b717-838d5b300f1c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.784836 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.784747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.784836 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:17.784805 2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 19:03:17.785017 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:17.784896 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-tls podName:d8a09f1e-f229-4f72-b717-838d5b300f1c nodeName:}" failed. No retries permitted until 2026-07-09 19:03:18.284874027 +0000 UTC m=+298.736326487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-tls") pod "prometheus-operator-64785b6675-wfphb" (UID: "d8a09f1e-f229-4f72-b717-838d5b300f1c") : secret "prometheus-operator-tls" not found Jul 09 19:03:17.785522 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.785482 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8a09f1e-f229-4f72-b717-838d5b300f1c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.787177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.787159 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.793725 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.793705 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-777kb\" (UniqueName: \"kubernetes.io/projected/d8a09f1e-f229-4f72-b717-838d5b300f1c-kube-api-access-777kb\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:17.920883 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:17.920778 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:18.289009 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:18.288974 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:18.291436 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:18.291411 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8a09f1e-f229-4f72-b717-838d5b300f1c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wfphb\" (UID: \"d8a09f1e-f229-4f72-b717-838d5b300f1c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:18.492418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:18.492374 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" Jul 09 19:03:18.621405 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:18.621374 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wfphb"] Jul 09 19:03:18.626992 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:18.626964 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a09f1e_f229_4f72_b717_838d5b300f1c.slice/crio-9290e3c9891d2b3ddbaf0372c679590ba52f8172fcfa0e1699997d41766242a8 WatchSource:0}: Error finding container 9290e3c9891d2b3ddbaf0372c679590ba52f8172fcfa0e1699997d41766242a8: Status 404 returned error can't find the container with id 9290e3c9891d2b3ddbaf0372c679590ba52f8172fcfa0e1699997d41766242a8 Jul 09 19:03:19.008763 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:19.008727 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" event={"ID":"d8a09f1e-f229-4f72-b717-838d5b300f1c","Type":"ContainerStarted","Data":"9290e3c9891d2b3ddbaf0372c679590ba52f8172fcfa0e1699997d41766242a8"} Jul 09 19:03:19.975749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:19.975716 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mc95l" Jul 09 19:03:19.983772 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:19.983743 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:03:19.986962 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:19.985927 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:03:19.992158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:19.992138 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 19:03:20.257620 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.257586 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:20.544835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.544751 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-997fbcf45-29925"] Jul 09 19:03:20.547858 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.547839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.556147 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.556120 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 19:03:20.557821 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.557798 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-997fbcf45-29925"] Jul 09 19:03:20.711141 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-serving-cert\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.711141 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711144 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42vbs\" (UniqueName: \"kubernetes.io/projected/1b9303bc-66e9-45fa-abe9-03375b1c8715-kube-api-access-42vbs\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.711401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711165 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-oauth-serving-cert\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.711401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-oauth-config\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.711401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711326 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-trusted-ca-bundle\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.711401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711396 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-config\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.711617 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.711437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-service-ca\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.755559 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.755485 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:20.812395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812311 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42vbs\" (UniqueName: \"kubernetes.io/projected/1b9303bc-66e9-45fa-abe9-03375b1c8715-kube-api-access-42vbs\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.812395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-oauth-serving-cert\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.812395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-oauth-config\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.812682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-trusted-ca-bundle\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.812682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-config\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.812682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812529 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-service-ca\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.812682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.812593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-serving-cert\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.813333 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.813305 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-config\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.813447 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.813335 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-service-ca\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.813575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.813551 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-oauth-serving-cert\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.813839 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.813819 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-trusted-ca-bundle\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.814462 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.814434 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:20.815426 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.815404 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-oauth-config\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.815523 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.815444 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-serving-cert\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.821029 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.821005 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42vbs\" (UniqueName: \"kubernetes.io/projected/1b9303bc-66e9-45fa-abe9-03375b1c8715-kube-api-access-42vbs\") pod \"console-997fbcf45-29925\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.857107 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.857073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:20.992381 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:20.992348 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-997fbcf45-29925"] Jul 09 19:03:20.995328 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:20.995303 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9303bc_66e9_45fa_abe9_03375b1c8715.slice/crio-87d2ae46eebf0394accb1aa5a5b9a35ea44f8678ba6f3c2424f1df9b1a5b2340 WatchSource:0}: Error finding container 87d2ae46eebf0394accb1aa5a5b9a35ea44f8678ba6f3c2424f1df9b1a5b2340: Status 404 returned error can't find the container with id 87d2ae46eebf0394accb1aa5a5b9a35ea44f8678ba6f3c2424f1df9b1a5b2340 Jul 09 19:03:21.019022 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:21.018988 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" event={"ID":"d8a09f1e-f229-4f72-b717-838d5b300f1c","Type":"ContainerStarted","Data":"2a6a4b86f52877f73ef36e126217910267bccd69dc0695655c7fbafa58b000ea"} Jul 09 19:03:21.019210 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:21.019057 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" event={"ID":"d8a09f1e-f229-4f72-b717-838d5b300f1c","Type":"ContainerStarted","Data":"814d097b5444e49db14e6ed1acd2bc12fff577ffbd12c1ee726fb55c090cf6ab"} Jul 09 19:03:21.020337 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:21.020313 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997fbcf45-29925" event={"ID":"1b9303bc-66e9-45fa-abe9-03375b1c8715","Type":"ContainerStarted","Data":"87d2ae46eebf0394accb1aa5a5b9a35ea44f8678ba6f3c2424f1df9b1a5b2340"} Jul 09 19:03:21.038041 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:21.037991 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-wfphb" podStartSLOduration=2.673895215 podStartE2EDuration="4.037975307s" totalImagesPullingTime="1.364080092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:21.036045479 +0000 UTC m=+301.487497940" watchObservedRunningTime="2026-07-09 19:03:21.037975307 +0000 UTC m=+301.489427766" Jul 09 19:03:22.025644 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.025606 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997fbcf45-29925" event={"ID":"1b9303bc-66e9-45fa-abe9-03375b1c8715","Type":"ContainerStarted","Data":"fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735"} Jul 09 19:03:22.042838 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.042791 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-997fbcf45-29925" podStartSLOduration=2.042774737 podStartE2EDuration="2.042774737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:22.042023146 +0000 UTC m=+302.493475607" watchObservedRunningTime="2026-07-09 19:03:22.042774737 +0000 UTC m=+302.494227216" Jul 09 19:03:22.272189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.272152 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:22.272391 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.272261 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:22.277135 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.277066 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:22.679044 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.678959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:22.893337 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.893305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:22.928838 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.928803 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6lcd7"] Jul 09 19:03:22.933562 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.933468 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:22.936857 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.936687 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 19:03:22.936857 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.936705 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 19:03:22.936857 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.936818 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fv76w\"" Jul 09 19:03:22.936857 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.936833 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 19:03:22.978404 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:22.978375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.030474 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-wtmp\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-root\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030605 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-sys\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x26fh\" (UniqueName: \"kubernetes.io/projected/ce2d9570-2634-4174-a950-c8488f5004c2-kube-api-access-x26fh\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030684 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030716 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-tls\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030740 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2d9570-2634-4174-a950-c8488f5004c2-metrics-client-ca\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.030952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.030771 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-textfile\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.035969 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.035941 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:23.039662 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.039632 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.131832 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.131801 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-sys\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.132016 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.131871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x26fh\" (UniqueName: \"kubernetes.io/projected/ce2d9570-2634-4174-a950-c8488f5004c2-kube-api-access-x26fh\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.132016 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.131954 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-sys\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.132213 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132186 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.132270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-tls\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.132270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132261 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2d9570-2634-4174-a950-c8488f5004c2-metrics-client-ca\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.132358 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-textfile\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132533 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-wtmp\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132808 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-wtmp\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132805 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-textfile\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132808 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-root\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.132862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.133000 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce2d9570-2634-4174-a950-c8488f5004c2-root\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.133023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.133006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce2d9570-2634-4174-a950-c8488f5004c2-metrics-client-ca\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.135022 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.134993 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-tls\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.135171 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.135155 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce2d9570-2634-4174-a950-c8488f5004c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.140343 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.140315 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.142976 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.142954 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x26fh\" (UniqueName: \"kubernetes.io/projected/ce2d9570-2634-4174-a950-c8488f5004c2-kube-api-access-x26fh\") pod \"node-exporter-6lcd7\" (UID: \"ce2d9570-2634-4174-a950-c8488f5004c2\") " pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.237624 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.237590 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.244440 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.244420 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6lcd7" Jul 09 19:03:23.256909 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:23.256881 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2d9570_2634_4174_a950_c8488f5004c2.slice/crio-3f7c41100916ed104b415ababa422757acb2a487897b4e6a6d6598ea2a28d970 WatchSource:0}: Error finding container 3f7c41100916ed104b415ababa422757acb2a487897b4e6a6d6598ea2a28d970: Status 404 returned error can't find the container with id 3f7c41100916ed104b415ababa422757acb2a487897b4e6a6d6598ea2a28d970 Jul 09 19:03:23.259255 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.259236 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:03:23.509897 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.509804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.610102 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.610075 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.913732 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.913636 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.986049 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.986014 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:23.995320 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.995290 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 19:03:23.999991 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:23.999964 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.002843 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.002819 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 19:03:24.002971 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.002863 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 19:03:24.002971 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.002821 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 19:03:24.003204 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003180 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 19:03:24.003314 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003235 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 19:03:24.003370 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003331 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 19:03:24.003424 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003371 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 19:03:24.003424 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003393 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 19:03:24.003541 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003455 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-d972l\"" Jul 09 19:03:24.003787 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.003772 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 19:03:24.015418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.015393 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 19:03:24.035023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.034935 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6lcd7" event={"ID":"ce2d9570-2634-4174-a950-c8488f5004c2","Type":"ContainerStarted","Data":"3f7c41100916ed104b415ababa422757acb2a487897b4e6a6d6598ea2a28d970"} Jul 09 19:03:24.140295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140252 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140298 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140564 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140325 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140564 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/795af1df-4fb9-4712-9c0d-8e50c35614c3-config-out\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140564 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140470 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140564 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140521 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140564 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/795af1df-4fb9-4712-9c0d-8e50c35614c3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140769 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140724 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140895 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140875 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.140973 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140942 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.141008 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140978 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-config-volume\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.141008 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.140996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-web-config\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.141089 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.141010 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvlbj\" (UniqueName: \"kubernetes.io/projected/795af1df-4fb9-4712-9c0d-8e50c35614c3-kube-api-access-lvlbj\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.240281 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.240250 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:24.242335 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242296 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/795af1df-4fb9-4712-9c0d-8e50c35614c3-config-out\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242347 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242374 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242399 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/795af1df-4fb9-4712-9c0d-8e50c35614c3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242487 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.242586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-config-volume\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.242749 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:24.242674 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-trusted-ca-bundle podName:795af1df-4fb9-4712-9c0d-8e50c35614c3 nodeName:}" failed. No retries permitted until 2026-07-09 19:03:24.742651433 +0000 UTC m=+305.194103881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "795af1df-4fb9-4712-9c0d-8e50c35614c3") : configmap references non-existent config key: ca-bundle.crt Jul 09 19:03:24.243121 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.243065 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.243407 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.243336 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-web-config\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.243407 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.243378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvlbj\" (UniqueName: \"kubernetes.io/projected/795af1df-4fb9-4712-9c0d-8e50c35614c3-kube-api-access-lvlbj\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.243593 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.243429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.243668 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.243602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.243730 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.243669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.244430 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.244402 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.246500 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.246475 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/795af1df-4fb9-4712-9c0d-8e50c35614c3-config-out\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.247169 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.246934 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.247463 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.247441 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.247982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.247590 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.247982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.247943 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.248264 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.248206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-config-volume\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.248370 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.248277 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.249875 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.249852 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/795af1df-4fb9-4712-9c0d-8e50c35614c3-web-config\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.250086 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.250065 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/795af1df-4fb9-4712-9c0d-8e50c35614c3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.251957 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.251940 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvlbj\" (UniqueName: \"kubernetes.io/projected/795af1df-4fb9-4712-9c0d-8e50c35614c3-kube-api-access-lvlbj\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.746894 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.746858 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.747808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.747781 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795af1df-4fb9-4712-9c0d-8e50c35614c3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"795af1df-4fb9-4712-9c0d-8e50c35614c3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.901651 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.901565 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-57ffb4c87-sfxhb"] Jul 09 19:03:24.910029 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.909995 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:24.912642 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.912618 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 19:03:24.912913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.912886 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 19:03:24.912913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.912886 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a9og2lm6bie8c\"" Jul 09 19:03:24.913099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.912892 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 19:03:24.913099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.912991 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 19:03:24.913099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.912890 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6d9g8\"" Jul 09 19:03:24.913254 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.913168 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 19:03:24.913407 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.913395 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 19:03:24.918604 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:24.918581 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57ffb4c87-sfxhb"] Jul 09 19:03:25.040313 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.040277 2561 generic.go:356] "Generic (PLEG): container finished" podID="ce2d9570-2634-4174-a950-c8488f5004c2" containerID="6c3856e41bca84cf01c707e132274ad991604bc5c253885e6fae55c81a005820" exitCode=0 Jul 09 19:03:25.040717 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.040367 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6lcd7" event={"ID":"ce2d9570-2634-4174-a950-c8488f5004c2","Type":"ContainerDied","Data":"6c3856e41bca84cf01c707e132274ad991604bc5c253885e6fae55c81a005820"} Jul 09 19:03:25.048848 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.048825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.048939 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.048865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc989\" (UniqueName: \"kubernetes.io/projected/95856664-6f2f-4aec-bfad-23bb25c553df-kube-api-access-cc989\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.048939 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.048892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-tls\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.049015 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.048948 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.049015 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.048980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95856664-6f2f-4aec-bfad-23bb25c553df-metrics-client-ca\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.049074 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.049017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.049074 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.049038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-grpc-tls\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.049138 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.049084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.055601 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.055526 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 19:03:25.058537 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:25.058491 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795af1df_4fb9_4712_9c0d_8e50c35614c3.slice/crio-7e3e7713e9694afbbde06a66ffb29b99ff1b2a453b2520dbcf8c3883504230ba WatchSource:0}: Error finding container 7e3e7713e9694afbbde06a66ffb29b99ff1b2a453b2520dbcf8c3883504230ba: Status 404 returned error can't find the container with id 7e3e7713e9694afbbde06a66ffb29b99ff1b2a453b2520dbcf8c3883504230ba Jul 09 19:03:25.149594 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149565 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.149745 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95856664-6f2f-4aec-bfad-23bb25c553df-metrics-client-ca\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.149809 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149741 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.149809 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-grpc-tls\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.149909 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149830 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.149909 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149877 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.150004 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc989\" (UniqueName: \"kubernetes.io/projected/95856664-6f2f-4aec-bfad-23bb25c553df-kube-api-access-cc989\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.150004 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.149995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-tls\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.151340 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.151276 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95856664-6f2f-4aec-bfad-23bb25c553df-metrics-client-ca\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.152972 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.152915 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.153682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.153663 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.153964 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.153937 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.153964 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.153957 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.154099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.153982 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-thanos-querier-tls\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.154099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.154041 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/95856664-6f2f-4aec-bfad-23bb25c553df-secret-grpc-tls\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.158377 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.158360 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc989\" (UniqueName: \"kubernetes.io/projected/95856664-6f2f-4aec-bfad-23bb25c553df-kube-api-access-cc989\") pod \"thanos-querier-57ffb4c87-sfxhb\" (UID: \"95856664-6f2f-4aec-bfad-23bb25c553df\") " pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.220353 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.220315 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:25.249740 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.249700 2561 patch_prober.go:28] interesting pod/image-registry-5f55989c55-9k6sd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 19:03:25.249881 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.249766 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 19:03:25.358052 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.358017 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57ffb4c87-sfxhb"] Jul 09 19:03:25.361779 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:25.361753 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95856664_6f2f_4aec_bfad_23bb25c553df.slice/crio-54085d7ac460585cd37f4b11beeaeb5e3befcfbe77dab0dcdf07f94bfafe7040 WatchSource:0}: Error finding container 54085d7ac460585cd37f4b11beeaeb5e3befcfbe77dab0dcdf07f94bfafe7040: Status 404 returned error can't find the container with id 54085d7ac460585cd37f4b11beeaeb5e3befcfbe77dab0dcdf07f94bfafe7040 Jul 09 19:03:25.843214 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.843130 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:25.974196 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:25.974137 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:26.046082 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.046051 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"54085d7ac460585cd37f4b11beeaeb5e3befcfbe77dab0dcdf07f94bfafe7040"} Jul 09 19:03:26.047250 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.047220 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"7e3e7713e9694afbbde06a66ffb29b99ff1b2a453b2520dbcf8c3883504230ba"} Jul 09 19:03:26.049332 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.049308 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6lcd7" event={"ID":"ce2d9570-2634-4174-a950-c8488f5004c2","Type":"ContainerStarted","Data":"abe5ff4a1cccb83223a42f7df24569b9a96f2a45e0affec2b186ab533da22b7c"} Jul 09 19:03:26.049415 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.049358 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6lcd7" event={"ID":"ce2d9570-2634-4174-a950-c8488f5004c2","Type":"ContainerStarted","Data":"c2b16519365ef5017f4bf0dfb8a87bfacf69404e1f25c87f567d5308f83b6ed2"} Jul 09 19:03:26.071380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.071168 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6lcd7" podStartSLOduration=2.61529333 podStartE2EDuration="4.071147157s" totalImagesPullingTime="1.38149584s" totalInitContainerRuntime="74.357987ms" isStatefulPod=true podCreationTimestamp="2026-07-09 19:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:26.068064726 +0000 UTC m=+306.519517189" watchObservedRunningTime="2026-07-09 19:03:26.071147157 +0000 UTC m=+306.522599618" Jul 09 19:03:26.422442 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.422408 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:26.960445 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:26.960407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:27.055498 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.055466 2561 generic.go:356] "Generic (PLEG): container finished" podID="795af1df-4fb9-4712-9c0d-8e50c35614c3" containerID="2ce9dceb248563818c40cddabb20a9efe7858d25b893befb164d65f0e8dbf518" exitCode=0 Jul 09 19:03:27.055936 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.055632 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerDied","Data":"2ce9dceb248563818c40cddabb20a9efe7858d25b893befb164d65f0e8dbf518"} Jul 09 19:03:27.087312 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.087275 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:27.184194 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.184156 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:27.284588 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.284548 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:27.342414 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.342370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:27.701311 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.701210 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:27.798769 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:27.798724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:28.025042 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:28.025006 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:28.094359 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:28.094319 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:03:29.065589 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.065548 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"bcecd7e3aae9f70a2cdcdb597c34120ffbb2faa9205758d24e9d92143e47e2cf"} Jul 09 19:03:29.065773 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.065643 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"c412d32bdf93ba621247a86719c13f6094d704b8d811bb1f9d5e672e064507f1"} Jul 09 19:03:29.065773 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.065682 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"761c31253ab264b05315c0efc56d48b45065b14d5f0ddbcd1b433fdc87a13b87"} Jul 09 19:03:29.068814 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.068780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"e8422f6016db18caedba65a1002d1539fb0c964decad26d7d13199c8b4566cda"} Jul 09 19:03:29.068992 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.068861 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"18a45d8d2c70ef4456a6068b89b3dd4f3f0d6f27bef81b625b35713e4d7d95a1"} Jul 09 19:03:29.068992 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.068898 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"91e27617b25ea6d8dff3d12ec9596b155ba2a26bcc0d66c24984326e50d34f7e"} Jul 09 19:03:29.068992 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.068932 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"61431db64492cf7e7f02f1efc45390dcc06fb4c4b77f17118ba763e32cb18847"} Jul 09 19:03:29.068992 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.068966 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"7fa103882a37c745b2670e39deb92c8723f2212d69954c8d612d408a1b6de9d0"} Jul 09 19:03:29.171290 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.171198 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 19:03:29.175747 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.175718 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.176576 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.176539 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:29.179578 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179242 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3h9g5s2tu9bl5\"" Jul 09 19:03:29.179578 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179454 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 19:03:29.179760 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179700 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 19:03:29.179760 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179716 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 19:03:29.179869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179777 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 19:03:29.179869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179718 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 19:03:29.179869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179805 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nqppq\"" Jul 09 19:03:29.179869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.179723 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 19:03:29.180150 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180104 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 19:03:29.180566 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180279 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 19:03:29.180566 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180314 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 19:03:29.180566 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180411 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 19:03:29.180566 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180557 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 19:03:29.180931 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180915 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 19:03:29.181009 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.180917 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 19:03:29.183398 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.183341 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 19:03:29.187453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.187434 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 19:03:29.192112 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.192091 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 19:03:29.194171 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gcxf\" (UniqueName: \"kubernetes.io/projected/3ddf050e-f986-4391-a075-ca379be9fa9a-kube-api-access-8gcxf\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194284 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194178 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194284 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194200 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194284 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194256 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194303 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194336 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194360 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194385 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ddf050e-f986-4391-a075-ca379be9fa9a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194407 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194445 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194500 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ddf050e-f986-4391-a075-ca379be9fa9a-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194607 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-config\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194635 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194694 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.194766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194754 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.195131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194812 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.195131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.194833 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.288561 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.288522 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:29.295644 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295598 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295693 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295722 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ddf050e-f986-4391-a075-ca379be9fa9a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295746 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ddf050e-f986-4391-a075-ca379be9fa9a-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.295835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295821 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295863 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-config\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295932 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.295980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296008 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296088 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296133 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gcxf\" (UniqueName: \"kubernetes.io/projected/3ddf050e-f986-4391-a075-ca379be9fa9a-kube-api-access-8gcxf\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296164 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296683 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296194 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.296683 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.296232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.297712 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.297682 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.298242 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.298301 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.299061 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.299531 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.300248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.301381 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.301495 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301943 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.301582 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ddf050e-f986-4391-a075-ca379be9fa9a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301943 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.301662 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ddf050e-f986-4391-a075-ca379be9fa9a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.301943 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.301667 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.302098 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.301948 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.302248 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.302227 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.302473 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.302444 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ddf050e-f986-4391-a075-ca379be9fa9a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.303691 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.303665 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-config\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.303796 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.303732 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.303881 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.303859 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.304030 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.304015 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ddf050e-f986-4391-a075-ca379be9fa9a-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.304431 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.304412 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ddf050e-f986-4391-a075-ca379be9fa9a-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.310427 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.310405 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gcxf\" (UniqueName: \"kubernetes.io/projected/3ddf050e-f986-4391-a075-ca379be9fa9a-kube-api-access-8gcxf\") pod \"prometheus-k8s-0\" (UID: \"3ddf050e-f986-4391-a075-ca379be9fa9a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.488808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.488763 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:29.670803 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:29.670764 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 19:03:29.677730 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:03:29.674445 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ddf050e_f986_4391_a075_ca379be9fa9a.slice/crio-4b73afd113144e2e40e8405f374d70700e40af6e066a7434c4d58c5d1960b02f WatchSource:0}: Error finding container 4b73afd113144e2e40e8405f374d70700e40af6e066a7434c4d58c5d1960b02f: Status 404 returned error can't find the container with id 4b73afd113144e2e40e8405f374d70700e40af6e066a7434c4d58c5d1960b02f Jul 09 19:03:30.077212 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.077185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"3b41bc8b4fea5150c6a8f9908cfe5a6f2b9158ba430f25f52c8a826a37968af4"} Jul 09 19:03:30.077340 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.077267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"b458bf324737683972d55cb44ded9a89331a71fe6bb569e75b5d049a482521e2"} Jul 09 19:03:30.080468 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.080442 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"795af1df-4fb9-4712-9c0d-8e50c35614c3","Type":"ContainerStarted","Data":"392f4028b93fe640893e574df7618961cca3025c9873162fa43d8e8f13be34e9"} Jul 09 19:03:30.082057 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.082026 2561 generic.go:356] "Generic (PLEG): container finished" podID="3ddf050e-f986-4391-a075-ca379be9fa9a" containerID="4e8e7ffba9d01b55bcc3f32a7234a47c6224e4eb0043a472f7feaefcb7035caa" exitCode=0 Jul 09 19:03:30.082205 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.082077 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerDied","Data":"4e8e7ffba9d01b55bcc3f32a7234a47c6224e4eb0043a472f7feaefcb7035caa"} Jul 09 19:03:30.082205 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.082143 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"4b73afd113144e2e40e8405f374d70700e40af6e066a7434c4d58c5d1960b02f"} Jul 09 19:03:30.112775 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.112724 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.23169527 podStartE2EDuration="7.112703505s" totalImagesPullingTime="3.864590238s" totalInitContainerRuntime="16.417997ms" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:30.109321879 +0000 UTC m=+310.560774363" watchObservedRunningTime="2026-07-09 19:03:30.112703505 +0000 UTC m=+310.564155957" Jul 09 19:03:30.573660 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.573618 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:30.674744 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.674701 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:30.857914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.857799 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:30.857914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.857923 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:30.862759 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:30.862732 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:31.088283 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.088243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" event={"ID":"95856664-6f2f-4aec-bfad-23bb25c553df","Type":"ContainerStarted","Data":"1d25330fc0bbecaa125281a1f8e0ce388e3b2a843ab96d7952dad092dd301435"} Jul 09 19:03:31.092139 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.092115 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:03:31.097985 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.097962 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f55989c55-9k6sd"] Jul 09 19:03:31.138671 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.138566 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" podStartSLOduration=2.787934174 podStartE2EDuration="7.138550249s" totalImagesPullingTime="4.350616075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:31.134181449 +0000 UTC m=+311.585633908" watchObservedRunningTime="2026-07-09 19:03:31.138550249 +0000 UTC m=+311.590002715" Jul 09 19:03:31.263816 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.263782 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-597f649fcf-4kqzp"] Jul 09 19:03:31.279804 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.279767 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:31.382578 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.382450 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:31.480075 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.479939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:31.778792 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.778757 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:31.879345 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:31.879311 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:32.103676 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.103590 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:32.593832 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.593795 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695cc89f96-7pxzr"] Jul 09 19:03:32.601330 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.601302 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.607741 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.607708 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695cc89f96-7pxzr"] Jul 09 19:03:32.630939 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.630903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-oauth-serving-cert\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.631101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.630965 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-serving-cert\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.631101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.631038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-service-ca\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.631195 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.631119 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-config\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.631195 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.631169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgqph\" (UniqueName: \"kubernetes.io/projected/345e09c0-247e-4536-9bf1-9bb9993ac25b-kube-api-access-zgqph\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.631273 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.631197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-trusted-ca-bundle\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.631319 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.631279 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-oauth-config\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732186 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-trusted-ca-bundle\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732341 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-oauth-config\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732341 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732296 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-oauth-serving-cert\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732404 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732363 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-serving-cert\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732463 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-service-ca\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732463 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-config\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.732579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.732493 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgqph\" (UniqueName: \"kubernetes.io/projected/345e09c0-247e-4536-9bf1-9bb9993ac25b-kube-api-access-zgqph\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.733127 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.733102 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-trusted-ca-bundle\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.733241 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.733133 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-service-ca\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.733241 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.733191 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-config\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.733308 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.733249 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-oauth-serving-cert\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.734852 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.734828 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-oauth-config\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.735019 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.734999 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-serving-cert\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.743303 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.743277 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgqph\" (UniqueName: \"kubernetes.io/projected/345e09c0-247e-4536-9bf1-9bb9993ac25b-kube-api-access-zgqph\") pod \"console-695cc89f96-7pxzr\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:32.918395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:32.918312 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:33.106221 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:33.106193 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-57ffb4c87-sfxhb" Jul 09 19:03:34.152092 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:34.152053 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695cc89f96-7pxzr"] Jul 09 19:03:34.840935 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:34.840839 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:34.898925 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:34.898892 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:35.116724 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.116620 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"445fc529c7081e6dbe45400c4752162e912ad8fe7086ba6bb79a9de2c719d374"} Jul 09 19:03:35.116724 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.116688 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"a9f40202787926336416a100d04c33b6716b6f7d0d1a16d6a43988a1b1eb9ac2"} Jul 09 19:03:35.116914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.116729 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"6c4eac3e8ef032efcaa69c0fb942d4b16d01bb7db278c11e4e34d82aea587f4d"} Jul 09 19:03:35.116914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.116763 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"0c357fee6d671c1df598a8ebdb3cc263515688b33a6db53b56b7f2df51e2f07b"} Jul 09 19:03:35.116914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.116791 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"31eaa2f2b22142addcc59d3ca3070920aa4054e31a50af3150269915a2b04897"} Jul 09 19:03:35.116914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.116823 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ddf050e-f986-4391-a075-ca379be9fa9a","Type":"ContainerStarted","Data":"76451d483b6fae02cf87635476443cfbadc14e1ecd722b46048febe57e513349"} Jul 09 19:03:35.118247 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.118214 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cc89f96-7pxzr" event={"ID":"345e09c0-247e-4536-9bf1-9bb9993ac25b","Type":"ContainerStarted","Data":"5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b"} Jul 09 19:03:35.118355 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.118251 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cc89f96-7pxzr" event={"ID":"345e09c0-247e-4536-9bf1-9bb9993ac25b","Type":"ContainerStarted","Data":"790d029eeb0054ad6e5532f4b0e9a51c66bf03ba331945781d4487943d56286d"} Jul 09 19:03:35.150531 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.150451 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.175690535 podStartE2EDuration="6.15043493s" totalImagesPullingTime="3.950882498s" totalInitContainerRuntime="23.861897ms" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:35.147685216 +0000 UTC m=+315.599137676" watchObservedRunningTime="2026-07-09 19:03:35.15043493 +0000 UTC m=+315.601887389" Jul 09 19:03:35.167764 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:35.167716 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695cc89f96-7pxzr" podStartSLOduration=3.167701024 podStartE2EDuration="3.167701024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:35.166462469 +0000 UTC m=+315.617914930" watchObservedRunningTime="2026-07-09 19:03:35.167701024 +0000 UTC m=+315.619153484" Jul 09 19:03:36.146562 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:36.146524 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:36.242190 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:36.242141 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:39.489779 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:39.489740 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:03:41.545277 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:41.545239 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:41.632243 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:41.632207 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:42.918890 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:42.918830 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:42.918890 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:42.918906 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:42.924012 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:42.923986 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:43.155190 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:43.155162 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:03:43.204266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:43.204181 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-997fbcf45-29925"] Jul 09 19:03:43.315393 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:43.315359 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:43.415706 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:43.415668 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:43.517302 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:43.517250 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:49.178469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:49.178387 2561 generic.go:356] "Generic (PLEG): container finished" podID="5d48a281-0903-416c-b93b-74557d5cb0cc" containerID="7e6d7d7bed16dc54f4f7ba7bddf78386484314cbf379ca05f7b1f89c5e331154" exitCode=0 Jul 09 19:03:49.178469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:49.178439 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" event={"ID":"5d48a281-0903-416c-b93b-74557d5cb0cc","Type":"ContainerDied","Data":"7e6d7d7bed16dc54f4f7ba7bddf78386484314cbf379ca05f7b1f89c5e331154"} Jul 09 19:03:49.178864 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:49.178777 2561 scope.go:117] "RemoveContainer" containerID="7e6d7d7bed16dc54f4f7ba7bddf78386484314cbf379ca05f7b1f89c5e331154" Jul 09 19:03:50.184797 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:50.184695 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-pjh5z" event={"ID":"5d48a281-0903-416c-b93b-74557d5cb0cc","Type":"ContainerStarted","Data":"512a86e677e95b938d809abd0277491d2e178ef7e8095a463cebd022d6c61984"} Jul 09 19:03:50.637559 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:50.637490 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:50.736828 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:50.736787 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:55.761302 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:55.761263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:55.827814 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:55.827784 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:56.125488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.125374 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" containerName="registry" containerID="cri-o://fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669" gracePeriod=30 Jul 09 19:03:56.294488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.294441 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-597f649fcf-4kqzp" podUID="849cdd66-6726-41a0-a945-3804ee1fa45d" containerName="console" containerID="cri-o://596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097" gracePeriod=15 Jul 09 19:03:56.417707 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.417681 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:56.471092 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471059 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vx6h\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-kube-api-access-8vx6h\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471254 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471112 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-installation-pull-secrets\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471254 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471169 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471254 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471218 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-image-registry-private-configuration\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471254 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471249 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9442778-7848-4690-a25a-882cf2c57aaf-ca-trust-extracted\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-registry-certificates\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471329 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-bound-sa-token\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.471456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.471387 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-trusted-ca\") pod \"a9442778-7848-4690-a25a-882cf2c57aaf\" (UID: \"a9442778-7848-4690-a25a-882cf2c57aaf\") " Jul 09 19:03:56.473651 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.472070 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:03:56.473651 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.472088 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:03:56.477953 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.477898 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:03:56.477953 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.477910 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:03:56.477953 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.477943 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-kube-api-access-8vx6h" (OuterVolumeSpecName: "kube-api-access-8vx6h") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "kube-api-access-8vx6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:03:56.479610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.479547 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:03:56.480721 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.480698 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:03:56.481949 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.481887 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9442778-7848-4690-a25a-882cf2c57aaf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a9442778-7848-4690-a25a-882cf2c57aaf" (UID: "a9442778-7848-4690-a25a-882cf2c57aaf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:03:56.538318 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.538291 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:56.572749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.572718 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-console-config\") pod \"849cdd66-6726-41a0-a945-3804ee1fa45d\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " Jul 09 19:03:56.572902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.572756 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-oauth-serving-cert\") pod \"849cdd66-6726-41a0-a945-3804ee1fa45d\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " Jul 09 19:03:56.572902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.572885 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-serving-cert\") pod \"849cdd66-6726-41a0-a945-3804ee1fa45d\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " Jul 09 19:03:56.573009 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.572934 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/849cdd66-6726-41a0-a945-3804ee1fa45d-kube-api-access-l8tz4\") pod \"849cdd66-6726-41a0-a945-3804ee1fa45d\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " Jul 09 19:03:56.573009 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.572966 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-oauth-config\") pod \"849cdd66-6726-41a0-a945-3804ee1fa45d\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " Jul 09 19:03:56.573108 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573048 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-service-ca\") pod \"849cdd66-6726-41a0-a945-3804ee1fa45d\" (UID: \"849cdd66-6726-41a0-a945-3804ee1fa45d\") " Jul 09 19:03:56.573213 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573165 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "849cdd66-6726-41a0-a945-3804ee1fa45d" (UID: "849cdd66-6726-41a0-a945-3804ee1fa45d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:03:56.573213 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573169 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-console-config" (OuterVolumeSpecName: "console-config") pod "849cdd66-6726-41a0-a945-3804ee1fa45d" (UID: "849cdd66-6726-41a0-a945-3804ee1fa45d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573396 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-bound-sa-token\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573416 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-trusted-ca\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573429 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vx6h\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-kube-api-access-8vx6h\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573445 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-installation-pull-secrets\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573458 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9442778-7848-4690-a25a-882cf2c57aaf-registry-tls\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573470 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-console-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573481 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-oauth-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573495 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9442778-7848-4690-a25a-882cf2c57aaf-image-registry-private-configuration\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573529 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9442778-7848-4690-a25a-882cf2c57aaf-ca-trust-extracted\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573543 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9442778-7848-4690-a25a-882cf2c57aaf-registry-certificates\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.573579 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.573540 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-service-ca" (OuterVolumeSpecName: "service-ca") pod "849cdd66-6726-41a0-a945-3804ee1fa45d" (UID: "849cdd66-6726-41a0-a945-3804ee1fa45d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:03:56.575252 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.575226 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "849cdd66-6726-41a0-a945-3804ee1fa45d" (UID: "849cdd66-6726-41a0-a945-3804ee1fa45d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:03:56.575759 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.575740 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "849cdd66-6726-41a0-a945-3804ee1fa45d" (UID: "849cdd66-6726-41a0-a945-3804ee1fa45d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:03:56.575829 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.575759 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849cdd66-6726-41a0-a945-3804ee1fa45d-kube-api-access-l8tz4" (OuterVolumeSpecName: "kube-api-access-l8tz4") pod "849cdd66-6726-41a0-a945-3804ee1fa45d" (UID: "849cdd66-6726-41a0-a945-3804ee1fa45d"). InnerVolumeSpecName "kube-api-access-l8tz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:03:56.674964 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.674877 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.674964 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.674905 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/849cdd66-6726-41a0-a945-3804ee1fa45d-kube-api-access-l8tz4\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.674964 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.674919 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/849cdd66-6726-41a0-a945-3804ee1fa45d-console-oauth-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:56.674964 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:56.674933 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/849cdd66-6726-41a0-a945-3804ee1fa45d-service-ca\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:03:57.215179 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.215148 2561 generic.go:356] "Generic (PLEG): container finished" podID="849cdd66-6726-41a0-a945-3804ee1fa45d" containerID="596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097" exitCode=0 Jul 09 19:03:57.215696 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.215195 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597f649fcf-4kqzp" Jul 09 19:03:57.215696 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.215248 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597f649fcf-4kqzp" event={"ID":"849cdd66-6726-41a0-a945-3804ee1fa45d","Type":"ContainerDied","Data":"596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097"} Jul 09 19:03:57.215696 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.215306 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597f649fcf-4kqzp" event={"ID":"849cdd66-6726-41a0-a945-3804ee1fa45d","Type":"ContainerDied","Data":"14c24ebeba78c0f52dc01dbf57654ac4f2c9927ef39b9d38b244ddd641a4d04c"} Jul 09 19:03:57.215696 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.215337 2561 scope.go:117] "RemoveContainer" containerID="596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097" Jul 09 19:03:57.216451 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.216427 2561 generic.go:356] "Generic (PLEG): container finished" podID="a9442778-7848-4690-a25a-882cf2c57aaf" containerID="fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669" exitCode=0 Jul 09 19:03:57.216584 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.216459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" event={"ID":"a9442778-7848-4690-a25a-882cf2c57aaf","Type":"ContainerDied","Data":"fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669"} Jul 09 19:03:57.216584 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.216482 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" Jul 09 19:03:57.216584 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.216492 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f55989c55-9k6sd" event={"ID":"a9442778-7848-4690-a25a-882cf2c57aaf","Type":"ContainerDied","Data":"1a459c484e79156a1f68af378fba421aef7a8d953d776157be33002fda93a000"} Jul 09 19:03:57.224390 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.224371 2561 scope.go:117] "RemoveContainer" containerID="596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097" Jul 09 19:03:57.224704 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:57.224682 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097\": container with ID starting with 596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097 not found: ID does not exist" containerID="596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097" Jul 09 19:03:57.224808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.224710 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097"} err="failed to get container status \"596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097\": rpc error: code = NotFound desc = could not find container \"596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097\": container with ID starting with 596edeeddd9305884832a35ac1077f69ea802c9f6f9fbeaa10638defb9054097 not found: ID does not exist" Jul 09 19:03:57.224808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.224751 2561 scope.go:117] "RemoveContainer" containerID="fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669" Jul 09 19:03:57.232818 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.232802 2561 scope.go:117] "RemoveContainer" containerID="fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669" Jul 09 19:03:57.233072 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:03:57.233051 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669\": container with ID starting with fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669 not found: ID does not exist" containerID="fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669" Jul 09 19:03:57.233135 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.233077 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669"} err="failed to get container status \"fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669\": rpc error: code = NotFound desc = could not find container \"fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669\": container with ID starting with fa49cb4eab81c27061a53c8879e504a16905f733441cbd4eb6d2cc47552bf669 not found: ID does not exist" Jul 09 19:03:57.242381 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.242360 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f55989c55-9k6sd"] Jul 09 19:03:57.245925 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.245906 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f55989c55-9k6sd"] Jul 09 19:03:57.256982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.256959 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-597f649fcf-4kqzp"] Jul 09 19:03:57.263106 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:57.263086 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-597f649fcf-4kqzp"] Jul 09 19:03:58.099993 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:58.099961 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849cdd66-6726-41a0-a945-3804ee1fa45d" path="/var/lib/kubelet/pods/849cdd66-6726-41a0-a945-3804ee1fa45d/volumes" Jul 09 19:03:58.100384 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:58.100369 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" path="/var/lib/kubelet/pods/a9442778-7848-4690-a25a-882cf2c57aaf/volumes" Jul 09 19:03:58.353602 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:58.353518 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:58.455839 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:03:58.455806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:00.236948 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:00.236913 2561 generic.go:356] "Generic (PLEG): container finished" podID="839a1ce0-a0bd-4a90-873d-91be28e94261" containerID="eeb8a85f5720b3a9acc2a6c3c7ce393b3a177c1dd1e2dd1ce43e134670c1b757" exitCode=0 Jul 09 19:04:00.237383 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:00.236964 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" event={"ID":"839a1ce0-a0bd-4a90-873d-91be28e94261","Type":"ContainerDied","Data":"eeb8a85f5720b3a9acc2a6c3c7ce393b3a177c1dd1e2dd1ce43e134670c1b757"} Jul 09 19:04:00.237682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:00.237657 2561 scope.go:117] "RemoveContainer" containerID="eeb8a85f5720b3a9acc2a6c3c7ce393b3a177c1dd1e2dd1ce43e134670c1b757" Jul 09 19:04:01.243219 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:01.243181 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wjndj" event={"ID":"839a1ce0-a0bd-4a90-873d-91be28e94261","Type":"ContainerStarted","Data":"104aa899bfbe8c4e7e41a2c285bcd0f743fbf90895e3a35a8903232c872f98dd"} Jul 09 19:04:08.233637 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.233553 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-997fbcf45-29925" podUID="1b9303bc-66e9-45fa-abe9-03375b1c8715" containerName="console" containerID="cri-o://fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735" gracePeriod=15 Jul 09 19:04:08.501028 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.500987 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:04:08.582863 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.582828 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-trusted-ca-bundle\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.582863 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.582863 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-config\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.583064 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.582905 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-serving-cert\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.583064 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583024 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42vbs\" (UniqueName: \"kubernetes.io/projected/1b9303bc-66e9-45fa-abe9-03375b1c8715-kube-api-access-42vbs\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.583152 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583068 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-oauth-config\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.583202 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583150 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-service-ca\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.583202 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583180 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-oauth-serving-cert\") pod \"1b9303bc-66e9-45fa-abe9-03375b1c8715\" (UID: \"1b9303bc-66e9-45fa-abe9-03375b1c8715\") " Jul 09 19:04:08.583291 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583223 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:04:08.583291 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583268 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-config" (OuterVolumeSpecName: "console-config") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:04:08.583533 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583489 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-trusted-ca-bundle\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.583653 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583541 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.583653 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583537 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:04:08.583653 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.583575 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:04:08.585314 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.585279 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:04:08.585408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.585315 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:04:08.585408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.585331 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9303bc-66e9-45fa-abe9-03375b1c8715-kube-api-access-42vbs" (OuterVolumeSpecName: "kube-api-access-42vbs") pod "1b9303bc-66e9-45fa-abe9-03375b1c8715" (UID: "1b9303bc-66e9-45fa-abe9-03375b1c8715"). InnerVolumeSpecName "kube-api-access-42vbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:04:08.684253 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.684219 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.684253 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.684249 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42vbs\" (UniqueName: \"kubernetes.io/projected/1b9303bc-66e9-45fa-abe9-03375b1c8715-kube-api-access-42vbs\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.684253 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.684259 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b9303bc-66e9-45fa-abe9-03375b1c8715-console-oauth-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.684475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.684267 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-service-ca\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.684475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:08.684275 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b9303bc-66e9-45fa-abe9-03375b1c8715-oauth-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:04:09.282651 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.282616 2561 generic.go:356] "Generic (PLEG): container finished" podID="1b9303bc-66e9-45fa-abe9-03375b1c8715" containerID="fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735" exitCode=0 Jul 09 19:04:09.283103 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.282669 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997fbcf45-29925" Jul 09 19:04:09.283103 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.282665 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997fbcf45-29925" event={"ID":"1b9303bc-66e9-45fa-abe9-03375b1c8715","Type":"ContainerDied","Data":"fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735"} Jul 09 19:04:09.283103 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.282809 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997fbcf45-29925" event={"ID":"1b9303bc-66e9-45fa-abe9-03375b1c8715","Type":"ContainerDied","Data":"87d2ae46eebf0394accb1aa5a5b9a35ea44f8678ba6f3c2424f1df9b1a5b2340"} Jul 09 19:04:09.283103 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.282843 2561 scope.go:117] "RemoveContainer" containerID="fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735" Jul 09 19:04:09.292161 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.292134 2561 scope.go:117] "RemoveContainer" containerID="fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735" Jul 09 19:04:09.292485 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:04:09.292463 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735\": container with ID starting with fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735 not found: ID does not exist" containerID="fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735" Jul 09 19:04:09.292655 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.292494 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735"} err="failed to get container status \"fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735\": rpc error: code = NotFound desc = could not find container \"fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735\": container with ID starting with fd4fee312fdf16813b495c23966a5344fe82d2a896e141d11b31b0f609a8d735 not found: ID does not exist" Jul 09 19:04:09.305441 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.305405 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-997fbcf45-29925"] Jul 09 19:04:09.309389 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:09.309362 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-997fbcf45-29925"] Jul 09 19:04:10.099469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:10.099431 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9303bc-66e9-45fa-abe9-03375b1c8715" path="/var/lib/kubelet/pods/1b9303bc-66e9-45fa-abe9-03375b1c8715/volumes" Jul 09 19:04:10.799195 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:10.799162 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:10.868833 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:10.868751 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:29.489271 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:29.489231 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:04:29.505602 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:29.505574 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:04:30.388029 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:30.388004 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 19:04:34.028091 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:34.028034 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:04:34.030628 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:34.030605 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f59493d-083f-4990-8df2-09f3c8845e86-metrics-certs\") pod \"network-metrics-daemon-stgpg\" (UID: \"3f59493d-083f-4990-8df2-09f3c8845e86\") " pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:04:34.098858 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:34.098829 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4jjw\"" Jul 09 19:04:34.105568 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:34.105547 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-stgpg" Jul 09 19:04:34.242662 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:34.242619 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-stgpg"] Jul 09 19:04:34.245995 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:04:34.245963 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f59493d_083f_4990_8df2_09f3c8845e86.slice/crio-22bebad1e6c56db57da8f553e2d59d7b04251d6d312b72ba53a15eb94f1928e1 WatchSource:0}: Error finding container 22bebad1e6c56db57da8f553e2d59d7b04251d6d312b72ba53a15eb94f1928e1: Status 404 returned error can't find the container with id 22bebad1e6c56db57da8f553e2d59d7b04251d6d312b72ba53a15eb94f1928e1 Jul 09 19:04:34.388848 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:34.388770 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-stgpg" event={"ID":"3f59493d-083f-4990-8df2-09f3c8845e86","Type":"ContainerStarted","Data":"22bebad1e6c56db57da8f553e2d59d7b04251d6d312b72ba53a15eb94f1928e1"} Jul 09 19:04:36.400204 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:36.400167 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-stgpg" event={"ID":"3f59493d-083f-4990-8df2-09f3c8845e86","Type":"ContainerStarted","Data":"476baa97bd57cd1cfd63c2fdb29435ae832d2d5518f32ac57f39597a2ccfcd25"} Jul 09 19:04:36.400613 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:36.400224 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-stgpg" event={"ID":"3f59493d-083f-4990-8df2-09f3c8845e86","Type":"ContainerStarted","Data":"07e0f9804f0716d668893d6a0ed34cb1a28d95e57b97d927a07ea63b760924c4"} Jul 09 19:04:36.417578 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:36.417495 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-stgpg" podStartSLOduration=375.308599845 podStartE2EDuration="6m16.417480421s" totalImagesPullingTime="1.108880576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:36.415405206 +0000 UTC m=+376.866857666" watchObservedRunningTime="2026-07-09 19:04:36.417480421 +0000 UTC m=+376.868932880" Jul 09 19:04:36.511600 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:36.511564 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:36.571801 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:36.571767 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:40.206114 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:40.206075 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:40.288213 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:40.288174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:42.173877 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:42.173811 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:42.423099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:42.423053 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:47.241429 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241389 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f8bd55d44-95sdk"] Jul 09 19:04:47.241920 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241867 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849cdd66-6726-41a0-a945-3804ee1fa45d" containerName="console" Jul 09 19:04:47.241920 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241883 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="849cdd66-6726-41a0-a945-3804ee1fa45d" containerName="console" Jul 09 19:04:47.241920 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241905 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9303bc-66e9-45fa-abe9-03375b1c8715" containerName="console" Jul 09 19:04:47.241920 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241912 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9303bc-66e9-45fa-abe9-03375b1c8715" containerName="console" Jul 09 19:04:47.242113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241936 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" containerName="registry" Jul 09 19:04:47.242113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.241944 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" containerName="registry" Jul 09 19:04:47.242113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.242015 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9442778-7848-4690-a25a-882cf2c57aaf" containerName="registry" Jul 09 19:04:47.242113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.242029 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="849cdd66-6726-41a0-a945-3804ee1fa45d" containerName="console" Jul 09 19:04:47.242113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.242038 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9303bc-66e9-45fa-abe9-03375b1c8715" containerName="console" Jul 09 19:04:47.251810 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.251781 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.256198 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.256171 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w9hbh\"" Jul 09 19:04:47.256447 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.256182 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 19:04:47.256581 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.256240 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 19:04:47.257217 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.257195 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 19:04:47.257861 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.257841 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 19:04:47.257971 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.257952 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 19:04:47.258786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.258763 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f8bd55d44-95sdk"] Jul 09 19:04:47.262611 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.262543 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 19:04:47.336698 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336664 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-metrics-client-ca\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.336873 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336716 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-telemeter-client-tls\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.336873 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336744 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-federate-client-tls\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.336873 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336834 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.336977 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336891 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.336977 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336925 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krc2n\" (UniqueName: \"kubernetes.io/projected/439e538c-a3ca-46ef-8ec9-4953a524ffaf-kube-api-access-krc2n\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.337038 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.336983 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-serving-certs-ca-bundle\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.337038 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.337027 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-secret-telemeter-client\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.437658 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-serving-certs-ca-bundle\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.437855 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437667 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-secret-telemeter-client\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.437855 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-metrics-client-ca\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.437855 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437753 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-telemeter-client-tls\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.438016 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-federate-client-tls\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.438016 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.438016 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437959 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.438016 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.437997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krc2n\" (UniqueName: \"kubernetes.io/projected/439e538c-a3ca-46ef-8ec9-4953a524ffaf-kube-api-access-krc2n\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.438424 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.438399 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-metrics-client-ca\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.438622 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.438409 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-serving-certs-ca-bundle\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.439155 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.439127 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/439e538c-a3ca-46ef-8ec9-4953a524ffaf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.440614 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.440592 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.440702 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.440614 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-telemeter-client-tls\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.440957 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.440935 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-federate-client-tls\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.441153 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.441129 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/439e538c-a3ca-46ef-8ec9-4953a524ffaf-secret-telemeter-client\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.447792 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.447771 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krc2n\" (UniqueName: \"kubernetes.io/projected/439e538c-a3ca-46ef-8ec9-4953a524ffaf-kube-api-access-krc2n\") pod \"telemeter-client-6f8bd55d44-95sdk\" (UID: \"439e538c-a3ca-46ef-8ec9-4953a524ffaf\") " pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.563797 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.563696 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" Jul 09 19:04:47.720756 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:47.720714 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f8bd55d44-95sdk"] Jul 09 19:04:47.725519 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:04:47.725464 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439e538c_a3ca_46ef_8ec9_4953a524ffaf.slice/crio-5526f0043a7b91eff73dd2b7edee352e3b210458cb75ab3370bfe194959ed344 WatchSource:0}: Error finding container 5526f0043a7b91eff73dd2b7edee352e3b210458cb75ab3370bfe194959ed344: Status 404 returned error can't find the container with id 5526f0043a7b91eff73dd2b7edee352e3b210458cb75ab3370bfe194959ed344 Jul 09 19:04:48.448135 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:48.448096 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" event={"ID":"439e538c-a3ca-46ef-8ec9-4953a524ffaf","Type":"ContainerStarted","Data":"5526f0043a7b91eff73dd2b7edee352e3b210458cb75ab3370bfe194959ed344"} Jul 09 19:04:49.834403 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:49.834365 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:49.906735 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:49.906698 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:50.457693 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:50.457603 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" event={"ID":"439e538c-a3ca-46ef-8ec9-4953a524ffaf","Type":"ContainerStarted","Data":"4dbbd88365448f91f6c486c39086ead99586280ef03f95b58379aa2da4d7927f"} Jul 09 19:04:50.457693 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:50.457660 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" event={"ID":"439e538c-a3ca-46ef-8ec9-4953a524ffaf","Type":"ContainerStarted","Data":"c723c64169f356ee62aa8fde84ea42c573794ba83a5b89e442301fdd827430a2"} Jul 09 19:04:50.457693 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:50.457688 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" event={"ID":"439e538c-a3ca-46ef-8ec9-4953a524ffaf","Type":"ContainerStarted","Data":"7fd0adc14bddf4f0f72812cbc68b721edcc9405527f9d1be105657bb31bff94e"} Jul 09 19:04:50.497148 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:50.497091 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f8bd55d44-95sdk" podStartSLOduration=1.056737447 podStartE2EDuration="3.497075546s" totalImagesPullingTime="2.440338099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:50.488033514 +0000 UTC m=+390.939485974" watchObservedRunningTime="2026-07-09 19:04:50.497075546 +0000 UTC m=+390.948528006" Jul 09 19:04:51.162447 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.162408 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-864878b79f-nm699"] Jul 09 19:04:51.164984 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.164964 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.186827 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.186804 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864878b79f-nm699"] Jul 09 19:04:51.229795 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.229768 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:51.271222 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271181 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-service-ca\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.271418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-serving-cert\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.271418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-config\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.271418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vwzz\" (UniqueName: \"kubernetes.io/projected/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-kube-api-access-2vwzz\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.271418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271322 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-oauth-config\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.271418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-trusted-ca-bundle\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.271418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.271409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-oauth-serving-cert\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.346387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.346353 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:51.372587 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-oauth-serving-cert\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.372750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372599 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-service-ca\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.372750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372639 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-serving-cert\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.372750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-config\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.372750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372675 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vwzz\" (UniqueName: \"kubernetes.io/projected/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-kube-api-access-2vwzz\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.372750 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372720 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-oauth-config\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.372991 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.372823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-trusted-ca-bundle\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.373387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.373343 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-oauth-serving-cert\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.373698 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.373426 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-service-ca\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.373698 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.373450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-config\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.373867 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.373840 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-trusted-ca-bundle\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.375472 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.375448 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-oauth-config\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.375580 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.375448 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-serving-cert\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.401116 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.401084 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vwzz\" (UniqueName: \"kubernetes.io/projected/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-kube-api-access-2vwzz\") pod \"console-864878b79f-nm699\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.474077 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.473975 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864878b79f-nm699" Jul 09 19:04:51.649840 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:51.649809 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864878b79f-nm699"] Jul 09 19:04:51.653266 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:04:51.653239 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfea6fd7_018f_4f38_a6a5_7dd319151ab0.slice/crio-d05982c0f197a160ab22bb97ea0f15ec540337215e6c3c249c03913baa25aedf WatchSource:0}: Error finding container d05982c0f197a160ab22bb97ea0f15ec540337215e6c3c249c03913baa25aedf: Status 404 returned error can't find the container with id d05982c0f197a160ab22bb97ea0f15ec540337215e6c3c249c03913baa25aedf Jul 09 19:04:52.468559 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:52.468526 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864878b79f-nm699" event={"ID":"cfea6fd7-018f-4f38-a6a5-7dd319151ab0","Type":"ContainerStarted","Data":"9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001"} Jul 09 19:04:52.468958 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:52.468577 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864878b79f-nm699" event={"ID":"cfea6fd7-018f-4f38-a6a5-7dd319151ab0","Type":"ContainerStarted","Data":"d05982c0f197a160ab22bb97ea0f15ec540337215e6c3c249c03913baa25aedf"} Jul 09 19:04:52.492206 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:52.492155 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-864878b79f-nm699" podStartSLOduration=1.4921349799999999 podStartE2EDuration="1.49213498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:52.491066707 +0000 UTC m=+392.942519167" watchObservedRunningTime="2026-07-09 19:04:52.49213498 +0000 UTC m=+392.943587440" Jul 09 19:04:57.444269 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:57.444231 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:57.541946 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:57.541867 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:59.709945 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.709914 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864878b79f-nm699"] Jul 09 19:04:59.745399 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.745366 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-855b7d5f54-p9b4c"] Jul 09 19:04:59.747653 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.747637 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.760493 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.760472 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855b7d5f54-p9b4c"] Jul 09 19:04:59.854610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-service-ca\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.854610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-oauth-config\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.854844 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854630 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-config\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.854844 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-oauth-serving-cert\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.854844 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854754 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-serving-cert\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.854844 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854786 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-trusted-ca-bundle\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.854844 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.854816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ctsk\" (UniqueName: \"kubernetes.io/projected/a1791fb2-eb37-47c3-b2d5-4c505e74803a-kube-api-access-6ctsk\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956060 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956020 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-service-ca\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956060 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-oauth-config\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956317 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956079 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-config\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956317 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-oauth-serving-cert\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956317 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-serving-cert\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956334 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-trusted-ca-bundle\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ctsk\" (UniqueName: \"kubernetes.io/projected/a1791fb2-eb37-47c3-b2d5-4c505e74803a-kube-api-access-6ctsk\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.956982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.956956 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-service-ca\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.957315 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.957037 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-config\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.957315 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.957131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-oauth-serving-cert\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.957465 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.957416 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-trusted-ca-bundle\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.958878 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.958855 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-oauth-config\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.959051 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.959036 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-serving-cert\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:04:59.966929 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:04:59.966868 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ctsk\" (UniqueName: \"kubernetes.io/projected/a1791fb2-eb37-47c3-b2d5-4c505e74803a-kube-api-access-6ctsk\") pod \"console-855b7d5f54-p9b4c\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:05:00.057094 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:00.057055 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:05:00.195449 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:00.195417 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855b7d5f54-p9b4c"] Jul 09 19:05:00.197752 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:05:00.197728 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1791fb2_eb37_47c3_b2d5_4c505e74803a.slice/crio-7b4eec83644fcecbf80faf2f1d425e76926392ffa3cffa94eae14b6dd7ed09f8 WatchSource:0}: Error finding container 7b4eec83644fcecbf80faf2f1d425e76926392ffa3cffa94eae14b6dd7ed09f8: Status 404 returned error can't find the container with id 7b4eec83644fcecbf80faf2f1d425e76926392ffa3cffa94eae14b6dd7ed09f8 Jul 09 19:05:00.505525 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:00.505459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b7d5f54-p9b4c" event={"ID":"a1791fb2-eb37-47c3-b2d5-4c505e74803a","Type":"ContainerStarted","Data":"012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f"} Jul 09 19:05:00.505722 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:00.505586 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b7d5f54-p9b4c" event={"ID":"a1791fb2-eb37-47c3-b2d5-4c505e74803a","Type":"ContainerStarted","Data":"7b4eec83644fcecbf80faf2f1d425e76926392ffa3cffa94eae14b6dd7ed09f8"} Jul 09 19:05:00.523189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:00.523129 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-855b7d5f54-p9b4c" podStartSLOduration=1.5231119469999999 podStartE2EDuration="1.523111947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:05:00.522324823 +0000 UTC m=+400.973777284" watchObservedRunningTime="2026-07-09 19:05:00.523111947 +0000 UTC m=+400.974564407" Jul 09 19:05:01.475010 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:01.474976 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-864878b79f-nm699" Jul 09 19:05:09.444023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:09.443967 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:09.501686 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:09.501652 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:10.057196 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:10.057158 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:05:10.057354 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:10.057215 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:05:10.062080 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:10.062057 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:05:10.554660 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:10.554629 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:05:10.602451 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:10.602417 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695cc89f96-7pxzr"] Jul 09 19:05:11.212130 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:11.212090 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:11.271354 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:11.271313 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:24.730444 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:24.730378 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-864878b79f-nm699" podUID="cfea6fd7-018f-4f38-a6a5-7dd319151ab0" containerName="console" containerID="cri-o://9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001" gracePeriod=15 Jul 09 19:05:24.988428 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:24.988404 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864878b79f-nm699" Jul 09 19:05:25.176418 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176386 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-service-ca\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176435 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-oauth-config\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176463 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-serving-cert\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176555 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-oauth-serving-cert\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176577 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-config\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176595 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vwzz\" (UniqueName: \"kubernetes.io/projected/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-kube-api-access-2vwzz\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176854 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176628 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-trusted-ca-bundle\") pod \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\" (UID: \"cfea6fd7-018f-4f38-a6a5-7dd319151ab0\") " Jul 09 19:05:25.176854 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176827 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-service-ca" (OuterVolumeSpecName: "service-ca") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:25.176940 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.176929 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-service-ca\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.177053 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.177022 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:25.177189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.177153 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-config" (OuterVolumeSpecName: "console-config") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:25.177269 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.177251 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:25.179004 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.178969 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:05:25.179114 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.179051 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-kube-api-access-2vwzz" (OuterVolumeSpecName: "kube-api-access-2vwzz") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "kube-api-access-2vwzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:05:25.179278 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.179260 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cfea6fd7-018f-4f38-a6a5-7dd319151ab0" (UID: "cfea6fd7-018f-4f38-a6a5-7dd319151ab0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:05:25.278177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.278087 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-oauth-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.278177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.278115 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.278177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.278146 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-oauth-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.278177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.278156 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-console-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.278177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.278165 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vwzz\" (UniqueName: \"kubernetes.io/projected/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-kube-api-access-2vwzz\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.278177 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.278176 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfea6fd7-018f-4f38-a6a5-7dd319151ab0-trusted-ca-bundle\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:25.615387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.615294 2561 generic.go:356] "Generic (PLEG): container finished" podID="cfea6fd7-018f-4f38-a6a5-7dd319151ab0" containerID="9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001" exitCode=0 Jul 09 19:05:25.615387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.615349 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864878b79f-nm699" Jul 09 19:05:25.615387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.615362 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864878b79f-nm699" event={"ID":"cfea6fd7-018f-4f38-a6a5-7dd319151ab0","Type":"ContainerDied","Data":"9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001"} Jul 09 19:05:25.615625 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.615436 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864878b79f-nm699" event={"ID":"cfea6fd7-018f-4f38-a6a5-7dd319151ab0","Type":"ContainerDied","Data":"d05982c0f197a160ab22bb97ea0f15ec540337215e6c3c249c03913baa25aedf"} Jul 09 19:05:25.615625 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.615458 2561 scope.go:117] "RemoveContainer" containerID="9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001" Jul 09 19:05:25.626924 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.626904 2561 scope.go:117] "RemoveContainer" containerID="9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001" Jul 09 19:05:25.627195 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:05:25.627175 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001\": container with ID starting with 9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001 not found: ID does not exist" containerID="9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001" Jul 09 19:05:25.627240 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.627206 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001"} err="failed to get container status \"9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001\": rpc error: code = NotFound desc = could not find container \"9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001\": container with ID starting with 9eefc03d29ddfeac3857385fabc76590801855fd63cc452a4ce9e11888036001 not found: ID does not exist" Jul 09 19:05:25.636726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.636703 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864878b79f-nm699"] Jul 09 19:05:25.639822 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.639794 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-864878b79f-nm699"] Jul 09 19:05:25.768965 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.768932 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:25.870494 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.870409 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:25.935324 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:25.935285 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:26.100305 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:26.100273 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfea6fd7-018f-4f38-a6a5-7dd319151ab0" path="/var/lib/kubelet/pods/cfea6fd7-018f-4f38-a6a5-7dd319151ab0/volumes" Jul 09 19:05:30.236688 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:30.236653 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:35.624688 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.624643 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-695cc89f96-7pxzr" podUID="345e09c0-247e-4536-9bf1-9bb9993ac25b" containerName="console" containerID="cri-o://5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b" gracePeriod=15 Jul 09 19:05:35.881878 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.881808 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:05:35.969482 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.969444 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-service-ca\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.969482 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.969482 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-config\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.969729 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.969547 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-trusted-ca-bundle\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.970488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.969963 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-oauth-serving-cert\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.970488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970037 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgqph\" (UniqueName: \"kubernetes.io/projected/345e09c0-247e-4536-9bf1-9bb9993ac25b-kube-api-access-zgqph\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.970488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970090 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-oauth-config\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.970488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970151 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-config" (OuterVolumeSpecName: "console-config") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:35.970488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970260 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-service-ca" (OuterVolumeSpecName: "service-ca") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:35.970488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970308 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:35.970885 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970562 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:05:35.970885 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.970727 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-serving-cert\") pod \"345e09c0-247e-4536-9bf1-9bb9993ac25b\" (UID: \"345e09c0-247e-4536-9bf1-9bb9993ac25b\") " Jul 09 19:05:35.971539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.971187 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-service-ca\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:35.971539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.971210 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:35.971539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.971225 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-trusted-ca-bundle\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:35.971539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.971239 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/345e09c0-247e-4536-9bf1-9bb9993ac25b-oauth-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:35.977234 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.977201 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345e09c0-247e-4536-9bf1-9bb9993ac25b-kube-api-access-zgqph" (OuterVolumeSpecName: "kube-api-access-zgqph") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "kube-api-access-zgqph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:05:35.977234 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.977225 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:05:35.977476 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:35.977461 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "345e09c0-247e-4536-9bf1-9bb9993ac25b" (UID: "345e09c0-247e-4536-9bf1-9bb9993ac25b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:05:36.071869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.071832 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:36.071869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.071863 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgqph\" (UniqueName: \"kubernetes.io/projected/345e09c0-247e-4536-9bf1-9bb9993ac25b-kube-api-access-zgqph\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:36.071869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.071872 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/345e09c0-247e-4536-9bf1-9bb9993ac25b-console-oauth-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:05:36.664802 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.664767 2561 generic.go:356] "Generic (PLEG): container finished" podID="345e09c0-247e-4536-9bf1-9bb9993ac25b" containerID="5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b" exitCode=0 Jul 09 19:05:36.665223 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.664830 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cc89f96-7pxzr" Jul 09 19:05:36.665223 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.664854 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cc89f96-7pxzr" event={"ID":"345e09c0-247e-4536-9bf1-9bb9993ac25b","Type":"ContainerDied","Data":"5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b"} Jul 09 19:05:36.665223 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.664906 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cc89f96-7pxzr" event={"ID":"345e09c0-247e-4536-9bf1-9bb9993ac25b","Type":"ContainerDied","Data":"790d029eeb0054ad6e5532f4b0e9a51c66bf03ba331945781d4487943d56286d"} Jul 09 19:05:36.665223 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.664928 2561 scope.go:117] "RemoveContainer" containerID="5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b" Jul 09 19:05:36.673589 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.673568 2561 scope.go:117] "RemoveContainer" containerID="5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b" Jul 09 19:05:36.673841 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:05:36.673822 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b\": container with ID starting with 5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b not found: ID does not exist" containerID="5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b" Jul 09 19:05:36.673886 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.673850 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b"} err="failed to get container status \"5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b\": rpc error: code = NotFound desc = could not find container \"5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b\": container with ID starting with 5798520c4ee3f0c10fcf5e12e6b8205ea4cfff39caed842c0f88510584561b9b not found: ID does not exist" Jul 09 19:05:36.683144 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.683121 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695cc89f96-7pxzr"] Jul 09 19:05:36.685346 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:36.685325 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695cc89f96-7pxzr"] Jul 09 19:05:38.099723 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:38.099691 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="345e09c0-247e-4536-9bf1-9bb9993ac25b" path="/var/lib/kubelet/pods/345e09c0-247e-4536-9bf1-9bb9993ac25b/volumes" Jul 09 19:05:46.516994 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:46.516894 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:05:46.614128 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:05:46.614089 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:08.682263 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:08.682220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:12.031609 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:12.031539 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:13.405647 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:13.405610 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:16.097320 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:16.097282 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:16.217967 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:16.217936 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:17.781223 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:17.781186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:17.943714 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:17.943679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:19.954149 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:19.954116 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:20.358500 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:20.358463 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:22.854957 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:22.854901 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:22.963266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:22.963214 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:28.367726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:28.367690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:30.314969 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:30.314928 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:30.415152 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:30.415123 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:36.205389 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:36.205352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:36.325283 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:36.325245 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:38.377278 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:38.377238 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:47.703059 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:47.703011 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:47.766914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:47.766878 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:52.854395 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:52.854348 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:52.963373 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:52.963345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:53.477897 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.477848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:53.738916 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.738878 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:53.881758 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.881726 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:53.917912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.917883 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd"] Jul 09 19:06:53.918255 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.918241 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfea6fd7-018f-4f38-a6a5-7dd319151ab0" containerName="console" Jul 09 19:06:53.918297 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.918258 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfea6fd7-018f-4f38-a6a5-7dd319151ab0" containerName="console" Jul 09 19:06:53.918297 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.918274 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345e09c0-247e-4536-9bf1-9bb9993ac25b" containerName="console" Jul 09 19:06:53.918297 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.918285 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="345e09c0-247e-4536-9bf1-9bb9993ac25b" containerName="console" Jul 09 19:06:53.918389 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.918366 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="345e09c0-247e-4536-9bf1-9bb9993ac25b" containerName="console" Jul 09 19:06:53.918389 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.918376 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfea6fd7-018f-4f38-a6a5-7dd319151ab0" containerName="console" Jul 09 19:06:53.921609 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.921592 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:53.924146 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.924125 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 19:06:53.924284 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.924172 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 19:06:53.924423 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.924410 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 19:06:53.924551 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.924496 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ffx4d\"" Jul 09 19:06:53.924703 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.924686 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 19:06:53.938729 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:53.938707 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd"] Jul 09 19:06:54.046913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.046817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cww9q\" (UniqueName: \"kubernetes.io/projected/1030534d-6078-49af-9143-88d95a426541-kube-api-access-cww9q\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.046913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.046858 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1030534d-6078-49af-9143-88d95a426541-apiservice-cert\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.046913 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.046892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1030534d-6078-49af-9143-88d95a426541-manifests\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.047173 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.046984 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1030534d-6078-49af-9143-88d95a426541-tmp\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.047173 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.047044 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1030534d-6078-49af-9143-88d95a426541-webhook-cert\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.061800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.061767 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:54.148082 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.148044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1030534d-6078-49af-9143-88d95a426541-webhook-cert\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.148270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.148100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cww9q\" (UniqueName: \"kubernetes.io/projected/1030534d-6078-49af-9143-88d95a426541-kube-api-access-cww9q\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.148270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.148132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1030534d-6078-49af-9143-88d95a426541-apiservice-cert\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.148270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.148171 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1030534d-6078-49af-9143-88d95a426541-manifests\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.148424 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.148322 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1030534d-6078-49af-9143-88d95a426541-tmp\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.148620 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.148601 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1030534d-6078-49af-9143-88d95a426541-manifests\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.150629 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.150606 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1030534d-6078-49af-9143-88d95a426541-tmp\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.150933 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.150916 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1030534d-6078-49af-9143-88d95a426541-apiservice-cert\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.150975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.150957 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1030534d-6078-49af-9143-88d95a426541-webhook-cert\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.159009 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.158985 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cww9q\" (UniqueName: \"kubernetes.io/projected/1030534d-6078-49af-9143-88d95a426541-kube-api-access-cww9q\") pod \"opendatahub-operator-controller-manager-799d55cdcd-7l5vd\" (UID: \"1030534d-6078-49af-9143-88d95a426541\") " pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.161946 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.161920 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:54.231591 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.231557 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:06:54.262419 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.262386 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:54.370012 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.369981 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd"] Jul 09 19:06:54.373488 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:06:54.373462 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1030534d_6078_49af_9143_88d95a426541.slice/crio-13a929d132d66a737e3631cbc9f305f8fc1a9bae027317a4440a1438d204269d WatchSource:0}: Error finding container 13a929d132d66a737e3631cbc9f305f8fc1a9bae027317a4440a1438d204269d: Status 404 returned error can't find the container with id 13a929d132d66a737e3631cbc9f305f8fc1a9bae027317a4440a1438d204269d Jul 09 19:06:54.374131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.374098 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:54.895880 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.895845 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:54.995792 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:54.995753 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" event={"ID":"1030534d-6078-49af-9143-88d95a426541","Type":"ContainerStarted","Data":"13a929d132d66a737e3631cbc9f305f8fc1a9bae027317a4440a1438d204269d"} Jul 09 19:06:57.166067 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:57.166027 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:58.013324 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:58.013280 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" event={"ID":"1030534d-6078-49af-9143-88d95a426541","Type":"ContainerStarted","Data":"1dac324b5e27783de1098584c573d50b89b0f5b283293c2fa00ef47a82a5a53e"} Jul 09 19:06:59.019433 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:59.019398 2561 generic.go:356] "Generic (PLEG): container finished" podID="1030534d-6078-49af-9143-88d95a426541" containerID="1dac324b5e27783de1098584c573d50b89b0f5b283293c2fa00ef47a82a5a53e" exitCode=0 Jul 09 19:06:59.019825 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:06:59.019442 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" event={"ID":"1030534d-6078-49af-9143-88d95a426541","Type":"ContainerDied","Data":"1dac324b5e27783de1098584c573d50b89b0f5b283293c2fa00ef47a82a5a53e"} Jul 09 19:07:00.025994 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:00.025951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" event={"ID":"1030534d-6078-49af-9143-88d95a426541","Type":"ContainerStarted","Data":"6599f53b53a9358fa34e18da485a6a81e9103b4917f261f6247cdeddfd0a493c"} Jul 09 19:07:00.026371 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:00.026043 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:07:00.052613 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:00.052574 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" podStartSLOduration=2.426541749 podStartE2EDuration="7.05255947s" totalImagesPullingTime="3.58507874s" totalInitContainerRuntime="1.040938981s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:00.051228612 +0000 UTC m=+520.502681083" watchObservedRunningTime="2026-07-09 19:07:00.05255947 +0000 UTC m=+520.504011929" Jul 09 19:07:02.633628 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:02.633591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:02.738656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:02.738621 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:10.941154 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.941072 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf"] Jul 09 19:07:10.950957 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.950935 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:10.954475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.954440 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 19:07:10.954475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.954441 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 19:07:10.954475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.954471 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 19:07:10.955871 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.955851 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 19:07:10.955994 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.955889 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tzt9f\"" Jul 09 19:07:10.956045 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.956001 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 19:07:10.960239 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.960215 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf"] Jul 09 19:07:10.992026 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.991996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26fddc1b-0e60-4db5-9ebf-558897e45396-cert\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:10.992168 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.992050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/26fddc1b-0e60-4db5-9ebf-558897e45396-manager-config\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:10.992168 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.992077 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrgh6\" (UniqueName: \"kubernetes.io/projected/26fddc1b-0e60-4db5-9ebf-558897e45396-kube-api-access-lrgh6\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:10.992168 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:10.992126 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/26fddc1b-0e60-4db5-9ebf-558897e45396-metrics-cert\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.037598 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.037570 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-799d55cdcd-7l5vd" Jul 09 19:07:11.093357 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.093322 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26fddc1b-0e60-4db5-9ebf-558897e45396-cert\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.093551 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.093425 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/26fddc1b-0e60-4db5-9ebf-558897e45396-manager-config\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.093551 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.093467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrgh6\" (UniqueName: \"kubernetes.io/projected/26fddc1b-0e60-4db5-9ebf-558897e45396-kube-api-access-lrgh6\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.093551 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.093531 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/26fddc1b-0e60-4db5-9ebf-558897e45396-metrics-cert\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.094265 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.094243 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/26fddc1b-0e60-4db5-9ebf-558897e45396-manager-config\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.096033 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.096006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26fddc1b-0e60-4db5-9ebf-558897e45396-cert\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.096143 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.096050 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/26fddc1b-0e60-4db5-9ebf-558897e45396-metrics-cert\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.103656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.103629 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrgh6\" (UniqueName: \"kubernetes.io/projected/26fddc1b-0e60-4db5-9ebf-558897e45396-kube-api-access-lrgh6\") pod \"lws-controller-manager-868778c6d9-2fzsf\" (UID: \"26fddc1b-0e60-4db5-9ebf-558897e45396\") " pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.260704 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.260666 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:11.393656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:11.393631 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf"] Jul 09 19:07:11.396518 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:07:11.396473 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26fddc1b_0e60_4db5_9ebf_558897e45396.slice/crio-c1c7eb3dae7d01589f201b48ea30dbdcee065fffa6298a182b38046b0d868e5d WatchSource:0}: Error finding container c1c7eb3dae7d01589f201b48ea30dbdcee065fffa6298a182b38046b0d868e5d: Status 404 returned error can't find the container with id c1c7eb3dae7d01589f201b48ea30dbdcee065fffa6298a182b38046b0d868e5d Jul 09 19:07:12.084392 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:12.084359 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" event={"ID":"26fddc1b-0e60-4db5-9ebf-558897e45396","Type":"ContainerStarted","Data":"c1c7eb3dae7d01589f201b48ea30dbdcee065fffa6298a182b38046b0d868e5d"} Jul 09 19:07:12.683397 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:12.683328 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:12.832047 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:12.832002 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:13.237349 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.237315 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:13.507472 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.507389 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-db49944d8-gdk2c"] Jul 09 19:07:13.516186 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.516154 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.528056 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.521089 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 19:07:13.528056 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.522538 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-44rlt\"" Jul 09 19:07:13.528056 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.522794 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 19:07:13.528056 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.524442 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-db49944d8-gdk2c"] Jul 09 19:07:13.620611 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.620572 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tls-certs\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.620611 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.620624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs5m2\" (UniqueName: \"kubernetes.io/projected/e1cdebdb-59c4-4e69-879c-4198a565cc8a-kube-api-access-gs5m2\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.620870 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.620741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tmp\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.721536 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.721474 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tmp\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.721801 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.721596 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tls-certs\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.721801 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.721631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs5m2\" (UniqueName: \"kubernetes.io/projected/e1cdebdb-59c4-4e69-879c-4198a565cc8a-kube-api-access-gs5m2\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.722413 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:07:13.722104 2561 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 09 19:07:13.722413 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:07:13.722194 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tls-certs podName:e1cdebdb-59c4-4e69-879c-4198a565cc8a nodeName:}" failed. No retries permitted until 2026-07-09 19:07:14.222173994 +0000 UTC m=+534.673626433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tls-certs") pod "kube-auth-proxy-db49944d8-gdk2c" (UID: "e1cdebdb-59c4-4e69-879c-4198a565cc8a") : secret "kube-auth-proxy-tls" not found Jul 09 19:07:13.724451 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.724424 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tmp\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.733980 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.733930 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs5m2\" (UniqueName: \"kubernetes.io/projected/e1cdebdb-59c4-4e69-879c-4198a565cc8a-kube-api-access-gs5m2\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:13.825283 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.825161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:13.928642 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:13.928603 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:14.065808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.065748 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:14.227376 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.227291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tls-certs\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:14.230228 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.230196 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1cdebdb-59c4-4e69-879c-4198a565cc8a-tls-certs\") pod \"kube-auth-proxy-db49944d8-gdk2c\" (UID: \"e1cdebdb-59c4-4e69-879c-4198a565cc8a\") " pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:14.257340 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.257285 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:14.436019 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.435982 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" Jul 09 19:07:14.544211 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.544177 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:14.600430 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.600405 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-db49944d8-gdk2c"] Jul 09 19:07:14.602839 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:07:14.602808 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1cdebdb_59c4_4e69_879c_4198a565cc8a.slice/crio-d14c17271863dafdadeb0da2ce2ac9282211368472087ff42238336759bdbca0 WatchSource:0}: Error finding container d14c17271863dafdadeb0da2ce2ac9282211368472087ff42238336759bdbca0: Status 404 returned error can't find the container with id d14c17271863dafdadeb0da2ce2ac9282211368472087ff42238336759bdbca0 Jul 09 19:07:14.643317 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.643286 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:14.743309 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.743262 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:14.898866 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:14.898835 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:15.101739 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:15.101635 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" event={"ID":"26fddc1b-0e60-4db5-9ebf-558897e45396","Type":"ContainerStarted","Data":"2ad3e4f4ef51b4b048b5e4c3ae85bee8721469de978fddfe8ca3b1fed383f045"} Jul 09 19:07:15.101739 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:15.101692 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:15.102804 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:15.102779 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" event={"ID":"e1cdebdb-59c4-4e69-879c-4198a565cc8a","Type":"ContainerStarted","Data":"d14c17271863dafdadeb0da2ce2ac9282211368472087ff42238336759bdbca0"} Jul 09 19:07:15.118501 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:15.118456 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" podStartSLOduration=2.05704066 podStartE2EDuration="5.118440023s" totalImagesPullingTime="3.061399363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:15.117778601 +0000 UTC m=+535.569231075" watchObservedRunningTime="2026-07-09 19:07:15.118440023 +0000 UTC m=+535.569892482" Jul 09 19:07:15.841640 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:15.841604 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:19.123010 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:19.122978 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" event={"ID":"e1cdebdb-59c4-4e69-879c-4198a565cc8a","Type":"ContainerStarted","Data":"b5803cf57078eea301482d9ec729a90928118c2787713aa786ec30c464052ffb"} Jul 09 19:07:19.146844 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:19.146793 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-db49944d8-gdk2c" podStartSLOduration=2.167474565 podStartE2EDuration="6.146777222s" totalImagesPullingTime="3.979302657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:19.144569642 +0000 UTC m=+539.596022101" watchObservedRunningTime="2026-07-09 19:07:19.146777222 +0000 UTC m=+539.598229729" Jul 09 19:07:21.791481 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:21.791442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:21.885864 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:21.885828 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:22.638147 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:22.638111 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:22.879746 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:22.879709 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:22.987160 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:22.987131 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:23.717291 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:23.717249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:23.821558 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:23.821521 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:23.898384 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:23.898342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:24.032182 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:24.032150 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:25.032420 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:25.032362 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:25.136883 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:25.136836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:25.232534 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:25.232473 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:26.109074 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:26.109040 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-868778c6d9-2fzsf" Jul 09 19:07:49.836451 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:49.836410 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:51.281597 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:51.281561 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:51.781587 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:51.781550 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:51.882106 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:07:51.882061 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:14.054629 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:14.054590 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:14.153199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:14.153168 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:14.281766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:14.281726 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:20.014079 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:20.014049 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:08:20.018471 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:20.018451 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:08:30.852946 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:30.852902 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:42.138846 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:42.138750 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:43.824051 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:43.824008 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:49.828639 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:49.828600 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:51.674233 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:51.674196 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:52.907824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:52.907789 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:53.038525 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:53.038474 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:53.863175 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:53.863135 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:54.871640 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:54.871606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:57.842974 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:57.842940 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:57.997468 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:57.997437 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw"] Jul 09 19:08:58.000584 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.000567 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.003347 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.003322 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 19:08:58.003493 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.003397 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 19:08:58.004776 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.004755 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 19:08:58.004776 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.004772 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4wjtj\"" Jul 09 19:08:58.004966 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.004759 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 19:08:58.016611 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.016254 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw"] Jul 09 19:08:58.022642 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.022613 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:58.086391 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.086360 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97420cc8-4422-4848-b14c-ed3e395fcb1b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.086583 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.086407 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pln6\" (UniqueName: \"kubernetes.io/projected/97420cc8-4422-4848-b14c-ed3e395fcb1b-kube-api-access-2pln6\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.086583 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.086442 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97420cc8-4422-4848-b14c-ed3e395fcb1b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.187465 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.187363 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97420cc8-4422-4848-b14c-ed3e395fcb1b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.187465 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.187440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97420cc8-4422-4848-b14c-ed3e395fcb1b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.187733 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.187488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pln6\" (UniqueName: \"kubernetes.io/projected/97420cc8-4422-4848-b14c-ed3e395fcb1b-kube-api-access-2pln6\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.187733 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:08:58.187631 2561 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 19:08:58.187733 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:08:58.187726 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97420cc8-4422-4848-b14c-ed3e395fcb1b-plugin-serving-cert podName:97420cc8-4422-4848-b14c-ed3e395fcb1b nodeName:}" failed. No retries permitted until 2026-07-09 19:08:58.687701432 +0000 UTC m=+639.139153873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/97420cc8-4422-4848-b14c-ed3e395fcb1b-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-65hvw" (UID: "97420cc8-4422-4848-b14c-ed3e395fcb1b") : secret "plugin-serving-cert" not found Jul 09 19:08:58.188122 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.188103 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97420cc8-4422-4848-b14c-ed3e395fcb1b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.196928 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.196896 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pln6\" (UniqueName: \"kubernetes.io/projected/97420cc8-4422-4848-b14c-ed3e395fcb1b-kube-api-access-2pln6\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.289634 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.289598 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:58.691316 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.691273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97420cc8-4422-4848-b14c-ed3e395fcb1b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.693861 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.693839 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97420cc8-4422-4848-b14c-ed3e395fcb1b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-65hvw\" (UID: \"97420cc8-4422-4848-b14c-ed3e395fcb1b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:58.922589 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:58.922548 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" Jul 09 19:08:59.056549 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:59.056498 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw"] Jul 09 19:08:59.059999 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:08:59.059964 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97420cc8_4422_4848_b14c_ed3e395fcb1b.slice/crio-534e63124d5ab9c1ae6860a2c11fb5dba98e80faa95786336d5461d6a3993ba9 WatchSource:0}: Error finding container 534e63124d5ab9c1ae6860a2c11fb5dba98e80faa95786336d5461d6a3993ba9: Status 404 returned error can't find the container with id 534e63124d5ab9c1ae6860a2c11fb5dba98e80faa95786336d5461d6a3993ba9 Jul 09 19:08:59.061185 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:59.061168 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:08:59.569334 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:08:59.569299 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" event={"ID":"97420cc8-4422-4848-b14c-ed3e395fcb1b","Type":"ContainerStarted","Data":"534e63124d5ab9c1ae6860a2c11fb5dba98e80faa95786336d5461d6a3993ba9"} Jul 09 19:09:00.291314 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.291276 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:00.323717 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.323685 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7"] Jul 09 19:09:00.327320 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.327293 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:00.330671 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.330644 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 19:09:00.331123 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.331102 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-gbmnj\"" Jul 09 19:09:00.346341 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.346317 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7"] Jul 09 19:09:00.434195 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.434025 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:00.509434 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.509388 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snt9z\" (UniqueName: \"kubernetes.io/projected/b63ef606-a837-47df-a988-90925356b044-kube-api-access-snt9z\") pod \"dns-operator-controller-manager-864dd78c4c-p8pb7\" (UID: \"b63ef606-a837-47df-a988-90925356b044\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:00.610132 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.610046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snt9z\" (UniqueName: \"kubernetes.io/projected/b63ef606-a837-47df-a988-90925356b044-kube-api-access-snt9z\") pod \"dns-operator-controller-manager-864dd78c4c-p8pb7\" (UID: \"b63ef606-a837-47df-a988-90925356b044\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:00.622559 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.622477 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snt9z\" (UniqueName: \"kubernetes.io/projected/b63ef606-a837-47df-a988-90925356b044-kube-api-access-snt9z\") pod \"dns-operator-controller-manager-864dd78c4c-p8pb7\" (UID: \"b63ef606-a837-47df-a988-90925356b044\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:00.642208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.642181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:00.805788 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:00.805755 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7"] Jul 09 19:09:00.810049 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:09:00.810005 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63ef606_a837_47df_a988_90925356b044.slice/crio-4fb150bbea4bd4de9efb95744d096ec4b8f2dd1a047be016b99be49ff6ff61e5 WatchSource:0}: Error finding container 4fb150bbea4bd4de9efb95744d096ec4b8f2dd1a047be016b99be49ff6ff61e5: Status 404 returned error can't find the container with id 4fb150bbea4bd4de9efb95744d096ec4b8f2dd1a047be016b99be49ff6ff61e5 Jul 09 19:09:01.584314 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:01.584243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" event={"ID":"b63ef606-a837-47df-a988-90925356b044","Type":"ContainerStarted","Data":"4fb150bbea4bd4de9efb95744d096ec4b8f2dd1a047be016b99be49ff6ff61e5"} Jul 09 19:09:01.878431 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:01.878345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:01.978500 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:01.978448 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:02.855414 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:02.855378 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:02.942365 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:02.942322 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:05.608718 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:05.608684 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" event={"ID":"b63ef606-a837-47df-a988-90925356b044","Type":"ContainerStarted","Data":"b98e549653e35097a03ff298f23d3550b6988fd3aa91e105d3c8aca07d62ae68"} Jul 09 19:09:05.609127 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:05.608832 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:05.627155 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:05.627092 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" podStartSLOduration=1.90629576 podStartE2EDuration="5.627072909s" totalImagesPullingTime="3.720777149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:05.626412005 +0000 UTC m=+646.077864460" watchObservedRunningTime="2026-07-09 19:09:05.627072909 +0000 UTC m=+646.078525369" Jul 09 19:09:10.104107 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:10.104068 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:10.206938 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:10.206896 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:14.866537 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:14.866481 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:14.936446 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:14.936387 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:16.617220 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:16.617176 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p8pb7" Jul 09 19:09:17.302378 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:17.302340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:17.399743 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:17.399697 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:22.697934 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:22.697901 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" event={"ID":"97420cc8-4422-4848-b14c-ed3e395fcb1b","Type":"ContainerStarted","Data":"5fec75b4f2f52acfdbbbfbf603f3120bed12fec005b3c6bfc75954ee94b951fd"} Jul 09 19:09:22.716416 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:22.716366 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-65hvw" podStartSLOduration=3.039445345 podStartE2EDuration="25.716350596s" totalImagesPullingTime="22.676905251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:22.714920223 +0000 UTC m=+663.166372676" watchObservedRunningTime="2026-07-09 19:09:22.716350596 +0000 UTC m=+663.167803056" Jul 09 19:09:22.852762 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:22.852733 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:22.957571 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:22.957462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:29.750413 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:29.750376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:29.851141 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:29.851105 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:34.570824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:34.570780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:34.718563 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:34.718499 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:37.002359 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:37.002313 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:37.102883 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:37.102855 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:39.437301 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:39.437256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:39.815046 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:39.815015 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:42.148023 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.147989 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:42.156458 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.156428 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:09:42.263580 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.263548 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:42.294035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.293999 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:09:42.294171 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.294046 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:09:42.294171 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.294126 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.297375 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.297357 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 19:09:42.317682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.317654 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djb62\" (UniqueName: \"kubernetes.io/projected/68b4ee10-6a74-4d39-89f1-f380771649d4-kube-api-access-djb62\") pod \"limitador-limitador-64c94fd64f-grkxn\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.317832 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.317716 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68b4ee10-6a74-4d39-89f1-f380771649d4-config-file\") pod \"limitador-limitador-64c94fd64f-grkxn\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.366806 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.366775 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:42.418235 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.418159 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djb62\" (UniqueName: \"kubernetes.io/projected/68b4ee10-6a74-4d39-89f1-f380771649d4-kube-api-access-djb62\") pod \"limitador-limitador-64c94fd64f-grkxn\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.418235 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.418223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68b4ee10-6a74-4d39-89f1-f380771649d4-config-file\") pod \"limitador-limitador-64c94fd64f-grkxn\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.418919 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.418900 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68b4ee10-6a74-4d39-89f1-f380771649d4-config-file\") pod \"limitador-limitador-64c94fd64f-grkxn\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.428663 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.428639 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djb62\" (UniqueName: \"kubernetes.io/projected/68b4ee10-6a74-4d39-89f1-f380771649d4-kube-api-access-djb62\") pod \"limitador-limitador-64c94fd64f-grkxn\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.604284 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.604245 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:42.760725 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.760697 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:42.768956 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.768933 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:09:42.771477 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:09:42.771447 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b4ee10_6a74_4d39_89f1_f380771649d4.slice/crio-40886968c9ce2aed0415db980aed5f3f7c976181ee7dec74dbadb70f83445b5f WatchSource:0}: Error finding container 40886968c9ce2aed0415db980aed5f3f7c976181ee7dec74dbadb70f83445b5f: Status 404 returned error can't find the container with id 40886968c9ce2aed0415db980aed5f3f7c976181ee7dec74dbadb70f83445b5f Jul 09 19:09:42.799863 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.799832 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" event={"ID":"68b4ee10-6a74-4d39-89f1-f380771649d4","Type":"ContainerStarted","Data":"40886968c9ce2aed0415db980aed5f3f7c976181ee7dec74dbadb70f83445b5f"} Jul 09 19:09:42.861551 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.861487 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:42.995966 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:42.995926 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:43.085675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.085642 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:43.089301 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.089275 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rc968"] Jul 09 19:09:43.115586 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.115554 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rc968"] Jul 09 19:09:43.115734 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.115668 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:09:43.118373 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.118352 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vfbr4\"" Jul 09 19:09:43.123971 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.123947 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcx48\" (UniqueName: \"kubernetes.io/projected/81764f8b-d8a3-4d10-97ef-345d0dd1c774-kube-api-access-gcx48\") pod \"authorino-75d556bd77-rc968\" (UID: \"81764f8b-d8a3-4d10-97ef-345d0dd1c774\") " pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:09:43.225071 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.225038 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcx48\" (UniqueName: \"kubernetes.io/projected/81764f8b-d8a3-4d10-97ef-345d0dd1c774-kube-api-access-gcx48\") pod \"authorino-75d556bd77-rc968\" (UID: \"81764f8b-d8a3-4d10-97ef-345d0dd1c774\") " pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:09:43.233628 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.233596 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcx48\" (UniqueName: \"kubernetes.io/projected/81764f8b-d8a3-4d10-97ef-345d0dd1c774-kube-api-access-gcx48\") pod \"authorino-75d556bd77-rc968\" (UID: \"81764f8b-d8a3-4d10-97ef-345d0dd1c774\") " pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:09:43.426239 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.426157 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:09:43.569473 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.569435 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rc968"] Jul 09 19:09:43.573673 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:09:43.573633 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81764f8b_d8a3_4d10_97ef_345d0dd1c774.slice/crio-089fd1188ca6b519bf73cfd2642fe704bb6a94d48290552a039cdd8e6f82be8d WatchSource:0}: Error finding container 089fd1188ca6b519bf73cfd2642fe704bb6a94d48290552a039cdd8e6f82be8d: Status 404 returned error can't find the container with id 089fd1188ca6b519bf73cfd2642fe704bb6a94d48290552a039cdd8e6f82be8d Jul 09 19:09:43.809584 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:43.809526 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rc968" event={"ID":"81764f8b-d8a3-4d10-97ef-345d0dd1c774","Type":"ContainerStarted","Data":"089fd1188ca6b519bf73cfd2642fe704bb6a94d48290552a039cdd8e6f82be8d"} Jul 09 19:09:49.853537 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:49.853479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" event={"ID":"68b4ee10-6a74-4d39-89f1-f380771649d4","Type":"ContainerStarted","Data":"be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95"} Jul 09 19:09:49.853995 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:49.853595 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:49.855212 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:49.855185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rc968" event={"ID":"81764f8b-d8a3-4d10-97ef-345d0dd1c774","Type":"ContainerStarted","Data":"fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06"} Jul 09 19:09:49.871533 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:49.871473 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" podStartSLOduration=1.5729000819999999 podStartE2EDuration="7.871458027s" totalImagesPullingTime="6.298557945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:49.869445984 +0000 UTC m=+690.320898445" watchObservedRunningTime="2026-07-09 19:09:49.871458027 +0000 UTC m=+690.322910531" Jul 09 19:09:49.884773 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:49.884722 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rc968" podStartSLOduration=1.388761531 podStartE2EDuration="6.884705365s" totalImagesPullingTime="5.495943834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:49.883154886 +0000 UTC m=+690.334607347" watchObservedRunningTime="2026-07-09 19:09:49.884705365 +0000 UTC m=+690.336157825" Jul 09 19:09:50.582706 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:50.582672 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:50.684120 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:50.684079 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:54.974922 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:54.974886 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:55.074078 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:55.074043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:56.462675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.462635 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dbb49779f-k7wlh"] Jul 09 19:09:56.518917 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.518874 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:56.524912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.524883 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbb49779f-k7wlh"] Jul 09 19:09:56.524996 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.524985 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658460 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658404 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-serving-cert\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658678 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658482 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-service-ca\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658678 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658539 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v4sk\" (UniqueName: \"kubernetes.io/projected/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-kube-api-access-4v4sk\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658678 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658609 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-oauth-config\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658678 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658638 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-trusted-ca-bundle\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658678 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-oauth-serving-cert\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.658871 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.658696 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-config\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.759408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759360 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-oauth-config\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.759408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759411 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-trusted-ca-bundle\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.759654 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759559 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-oauth-serving-cert\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.759654 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-config\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.759734 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759663 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-serving-cert\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.759867 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-service-ca\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.760031 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.759888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v4sk\" (UniqueName: \"kubernetes.io/projected/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-kube-api-access-4v4sk\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.760268 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.760245 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-oauth-serving-cert\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.760360 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.760254 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-trusted-ca-bundle\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.760360 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.760300 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-config\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.760479 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.760456 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-service-ca\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.762672 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.762653 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-oauth-config\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.762858 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.762838 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-console-serving-cert\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.768863 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.768839 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v4sk\" (UniqueName: \"kubernetes.io/projected/4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a-kube-api-access-4v4sk\") pod \"console-6dbb49779f-k7wlh\" (UID: \"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a\") " pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.800341 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.800292 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:56.834927 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.834884 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:09:56.978778 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:56.978746 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbb49779f-k7wlh"] Jul 09 19:09:56.981440 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:09:56.981410 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fffcff8_0b8e_4605_a31d_00c8cb2f5d0a.slice/crio-571471bc7b425786176514cada5f77266227a222b3eb14161d583e334fdda667 WatchSource:0}: Error finding container 571471bc7b425786176514cada5f77266227a222b3eb14161d583e334fdda667: Status 404 returned error can't find the container with id 571471bc7b425786176514cada5f77266227a222b3eb14161d583e334fdda667 Jul 09 19:09:57.416035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:57.415942 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:57.468801 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:57.468764 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:09:57.897265 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:57.897227 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbb49779f-k7wlh" event={"ID":"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a","Type":"ContainerStarted","Data":"dcc815991fc4d9005afd267b6ac3d30a97208678fe183beee930355a84c2145a"} Jul 09 19:09:57.897476 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:57.897278 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbb49779f-k7wlh" event={"ID":"4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a","Type":"ContainerStarted","Data":"571471bc7b425786176514cada5f77266227a222b3eb14161d583e334fdda667"} Jul 09 19:09:57.917158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:57.917099 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dbb49779f-k7wlh" podStartSLOduration=1.917076957 podStartE2EDuration="1.917076957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:57.915154206 +0000 UTC m=+698.366606679" watchObservedRunningTime="2026-07-09 19:09:57.917076957 +0000 UTC m=+698.368529468" Jul 09 19:09:59.010907 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.010867 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:09:59.011386 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.011138 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" podUID="68b4ee10-6a74-4d39-89f1-f380771649d4" containerName="limitador" containerID="cri-o://be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95" gracePeriod=30 Jul 09 19:09:59.012961 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.012940 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:59.458355 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.458330 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:59.587219 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.587116 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68b4ee10-6a74-4d39-89f1-f380771649d4-config-file\") pod \"68b4ee10-6a74-4d39-89f1-f380771649d4\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " Jul 09 19:09:59.587219 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.587191 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djb62\" (UniqueName: \"kubernetes.io/projected/68b4ee10-6a74-4d39-89f1-f380771649d4-kube-api-access-djb62\") pod \"68b4ee10-6a74-4d39-89f1-f380771649d4\" (UID: \"68b4ee10-6a74-4d39-89f1-f380771649d4\") " Jul 09 19:09:59.587577 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.587554 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b4ee10-6a74-4d39-89f1-f380771649d4-config-file" (OuterVolumeSpecName: "config-file") pod "68b4ee10-6a74-4d39-89f1-f380771649d4" (UID: "68b4ee10-6a74-4d39-89f1-f380771649d4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:09:59.589589 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.589568 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b4ee10-6a74-4d39-89f1-f380771649d4-kube-api-access-djb62" (OuterVolumeSpecName: "kube-api-access-djb62") pod "68b4ee10-6a74-4d39-89f1-f380771649d4" (UID: "68b4ee10-6a74-4d39-89f1-f380771649d4"). InnerVolumeSpecName "kube-api-access-djb62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:09:59.688879 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.688827 2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68b4ee10-6a74-4d39-89f1-f380771649d4-config-file\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:09:59.688879 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.688870 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djb62\" (UniqueName: \"kubernetes.io/projected/68b4ee10-6a74-4d39-89f1-f380771649d4-kube-api-access-djb62\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:09:59.908296 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.908198 2561 generic.go:356] "Generic (PLEG): container finished" podID="68b4ee10-6a74-4d39-89f1-f380771649d4" containerID="be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95" exitCode=0 Jul 09 19:09:59.908296 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.908253 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" Jul 09 19:09:59.908296 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.908264 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" event={"ID":"68b4ee10-6a74-4d39-89f1-f380771649d4","Type":"ContainerDied","Data":"be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95"} Jul 09 19:09:59.908599 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.908310 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-grkxn" event={"ID":"68b4ee10-6a74-4d39-89f1-f380771649d4","Type":"ContainerDied","Data":"40886968c9ce2aed0415db980aed5f3f7c976181ee7dec74dbadb70f83445b5f"} Jul 09 19:09:59.908599 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.908333 2561 scope.go:117] "RemoveContainer" containerID="be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95" Jul 09 19:09:59.917354 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.917335 2561 scope.go:117] "RemoveContainer" containerID="be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95" Jul 09 19:09:59.917717 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:09:59.917689 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95\": container with ID starting with be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95 not found: ID does not exist" containerID="be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95" Jul 09 19:09:59.917821 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.917722 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95"} err="failed to get container status \"be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95\": rpc error: code = NotFound desc = could not find container \"be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95\": container with ID starting with be9a3ca21f992898cf026812695300148e30b38e4ac6e66040aa1520e5031d95 not found: ID does not exist" Jul 09 19:09:59.928869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.928842 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:09:59.932288 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:09:59.932267 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-grkxn"] Jul 09 19:10:00.100968 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:00.100933 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b4ee10-6a74-4d39-89f1-f380771649d4" path="/var/lib/kubelet/pods/68b4ee10-6a74-4d39-89f1-f380771649d4/volumes" Jul 09 19:10:03.351624 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.351580 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:03.452110 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.452077 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:03.799231 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.799182 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6"] Jul 09 19:10:03.799824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.799802 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b4ee10-6a74-4d39-89f1-f380771649d4" containerName="limitador" Jul 09 19:10:03.799904 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.799830 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b4ee10-6a74-4d39-89f1-f380771649d4" containerName="limitador" Jul 09 19:10:03.799968 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.799948 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b4ee10-6a74-4d39-89f1-f380771649d4" containerName="limitador" Jul 09 19:10:03.804937 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.804912 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:03.808125 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.808080 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 19:10:03.808309 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.808141 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 19:10:03.808309 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.808228 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 19:10:03.808463 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.808340 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-p7cjd\"" Jul 09 19:10:03.809140 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.809112 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6"] Jul 09 19:10:03.931721 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.931665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr7h8\" (UniqueName: \"kubernetes.io/projected/3cba549a-60bc-4a35-a5f8-09b405c05596-kube-api-access-lr7h8\") pod \"postgres-69fb4594cf-5lsg6\" (UID: \"3cba549a-60bc-4a35-a5f8-09b405c05596\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:03.931908 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.931753 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3cba549a-60bc-4a35-a5f8-09b405c05596-data\") pod \"postgres-69fb4594cf-5lsg6\" (UID: \"3cba549a-60bc-4a35-a5f8-09b405c05596\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:03.958482 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:03.958449 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:04.032801 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.032768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr7h8\" (UniqueName: \"kubernetes.io/projected/3cba549a-60bc-4a35-a5f8-09b405c05596-kube-api-access-lr7h8\") pod \"postgres-69fb4594cf-5lsg6\" (UID: \"3cba549a-60bc-4a35-a5f8-09b405c05596\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:04.033026 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.032824 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3cba549a-60bc-4a35-a5f8-09b405c05596-data\") pod \"postgres-69fb4594cf-5lsg6\" (UID: \"3cba549a-60bc-4a35-a5f8-09b405c05596\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:04.033259 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.033238 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3cba549a-60bc-4a35-a5f8-09b405c05596-data\") pod \"postgres-69fb4594cf-5lsg6\" (UID: \"3cba549a-60bc-4a35-a5f8-09b405c05596\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:04.042409 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.042383 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr7h8\" (UniqueName: \"kubernetes.io/projected/3cba549a-60bc-4a35-a5f8-09b405c05596-kube-api-access-lr7h8\") pod \"postgres-69fb4594cf-5lsg6\" (UID: \"3cba549a-60bc-4a35-a5f8-09b405c05596\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:04.116063 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.115967 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:04.266975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.266949 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6"] Jul 09 19:10:04.269441 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:10:04.269413 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cba549a_60bc_4a35_a5f8_09b405c05596.slice/crio-11dd404e5f67813eaeeb78a577025013c738c7826a39791e595cce52ea14ffbd WatchSource:0}: Error finding container 11dd404e5f67813eaeeb78a577025013c738c7826a39791e595cce52ea14ffbd: Status 404 returned error can't find the container with id 11dd404e5f67813eaeeb78a577025013c738c7826a39791e595cce52ea14ffbd Jul 09 19:10:04.934624 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:04.934583 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" event={"ID":"3cba549a-60bc-4a35-a5f8-09b405c05596","Type":"ContainerStarted","Data":"11dd404e5f67813eaeeb78a577025013c738c7826a39791e595cce52ea14ffbd"} Jul 09 19:10:06.835725 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:06.835689 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:10:06.836251 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:06.835785 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:10:06.842432 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:06.842407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:10:06.951686 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:06.951652 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dbb49779f-k7wlh" Jul 09 19:10:06.996849 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:06.996813 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855b7d5f54-p9b4c"] Jul 09 19:10:07.091911 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:07.091805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:07.193211 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:07.193160 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:07.247691 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:07.247656 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:09.965867 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:09.965783 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" event={"ID":"3cba549a-60bc-4a35-a5f8-09b405c05596","Type":"ContainerStarted","Data":"997dae33cd5e6800a9418cb77c1dc71af54a820c2344d5f789b79cff331c4724"} Jul 09 19:10:09.966214 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:09.965913 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:09.983223 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:09.983171 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" podStartSLOduration=1.6397248260000001 podStartE2EDuration="6.98315458s" totalImagesPullingTime="5.343429754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:10:09.982624044 +0000 UTC m=+710.434076504" watchObservedRunningTime="2026-07-09 19:10:09.98315458 +0000 UTC m=+710.434607040" Jul 09 19:10:16.002904 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.002819 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5lsg6" Jul 09 19:10:16.305644 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.305535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:16.405486 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.405417 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:16.523975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.523936 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fhtjm"] Jul 09 19:10:16.527282 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.527265 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:16.534832 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.534810 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fhtjm"] Jul 09 19:10:16.558643 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.558553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpd8\" (UniqueName: \"kubernetes.io/projected/932125e5-5bfa-4d5b-82ce-201ca0fa76bd-kube-api-access-jzpd8\") pod \"authorino-5d4b67c446-fhtjm\" (UID: \"932125e5-5bfa-4d5b-82ce-201ca0fa76bd\") " pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:16.659765 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.659726 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpd8\" (UniqueName: \"kubernetes.io/projected/932125e5-5bfa-4d5b-82ce-201ca0fa76bd-kube-api-access-jzpd8\") pod \"authorino-5d4b67c446-fhtjm\" (UID: \"932125e5-5bfa-4d5b-82ce-201ca0fa76bd\") " pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:16.668983 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.668953 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpd8\" (UniqueName: \"kubernetes.io/projected/932125e5-5bfa-4d5b-82ce-201ca0fa76bd-kube-api-access-jzpd8\") pod \"authorino-5d4b67c446-fhtjm\" (UID: \"932125e5-5bfa-4d5b-82ce-201ca0fa76bd\") " pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:16.723644 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.723601 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fhtjm"] Jul 09 19:10:16.723898 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.723885 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:16.862763 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:16.862679 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fhtjm"] Jul 09 19:10:16.864997 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:10:16.864965 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod932125e5_5bfa_4d5b_82ce_201ca0fa76bd.slice/crio-06391c8e29ad879646036714fdad407052208728704254b51170ced4f43d1865 WatchSource:0}: Error finding container 06391c8e29ad879646036714fdad407052208728704254b51170ced4f43d1865: Status 404 returned error can't find the container with id 06391c8e29ad879646036714fdad407052208728704254b51170ced4f43d1865 Jul 09 19:10:17.001256 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:17.001215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" event={"ID":"932125e5-5bfa-4d5b-82ce-201ca0fa76bd","Type":"ContainerStarted","Data":"06391c8e29ad879646036714fdad407052208728704254b51170ced4f43d1865"} Jul 09 19:10:18.006931 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.006897 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" event={"ID":"932125e5-5bfa-4d5b-82ce-201ca0fa76bd","Type":"ContainerStarted","Data":"a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849"} Jul 09 19:10:18.007334 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.006998 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" podUID="932125e5-5bfa-4d5b-82ce-201ca0fa76bd" containerName="authorino" containerID="cri-o://a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849" gracePeriod=30 Jul 09 19:10:18.022724 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.022681 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" podStartSLOduration=1.443978347 podStartE2EDuration="2.022664839s" totalImagesPullingTime="578.686492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:10:18.021650028 +0000 UTC m=+718.473102504" watchObservedRunningTime="2026-07-09 19:10:18.022664839 +0000 UTC m=+718.474117320" Jul 09 19:10:18.171007 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.170977 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rc968"] Jul 09 19:10:18.171168 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.171138 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rc968" podUID="81764f8b-d8a3-4d10-97ef-345d0dd1c774" containerName="authorino" containerID="cri-o://fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06" gracePeriod=30 Jul 09 19:10:18.301365 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.301338 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:18.371102 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.371065 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzpd8\" (UniqueName: \"kubernetes.io/projected/932125e5-5bfa-4d5b-82ce-201ca0fa76bd-kube-api-access-jzpd8\") pod \"932125e5-5bfa-4d5b-82ce-201ca0fa76bd\" (UID: \"932125e5-5bfa-4d5b-82ce-201ca0fa76bd\") " Jul 09 19:10:18.373195 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.373169 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932125e5-5bfa-4d5b-82ce-201ca0fa76bd-kube-api-access-jzpd8" (OuterVolumeSpecName: "kube-api-access-jzpd8") pod "932125e5-5bfa-4d5b-82ce-201ca0fa76bd" (UID: "932125e5-5bfa-4d5b-82ce-201ca0fa76bd"). InnerVolumeSpecName "kube-api-access-jzpd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:10:18.410764 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.410738 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:10:18.472733 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.472691 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzpd8\" (UniqueName: \"kubernetes.io/projected/932125e5-5bfa-4d5b-82ce-201ca0fa76bd-kube-api-access-jzpd8\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:18.573122 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.573038 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcx48\" (UniqueName: \"kubernetes.io/projected/81764f8b-d8a3-4d10-97ef-345d0dd1c774-kube-api-access-gcx48\") pod \"81764f8b-d8a3-4d10-97ef-345d0dd1c774\" (UID: \"81764f8b-d8a3-4d10-97ef-345d0dd1c774\") " Jul 09 19:10:18.575201 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.575175 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81764f8b-d8a3-4d10-97ef-345d0dd1c774-kube-api-access-gcx48" (OuterVolumeSpecName: "kube-api-access-gcx48") pod "81764f8b-d8a3-4d10-97ef-345d0dd1c774" (UID: "81764f8b-d8a3-4d10-97ef-345d0dd1c774"). InnerVolumeSpecName "kube-api-access-gcx48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:10:18.674621 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:18.674584 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcx48\" (UniqueName: \"kubernetes.io/projected/81764f8b-d8a3-4d10-97ef-345d0dd1c774-kube-api-access-gcx48\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:19.012726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.012691 2561 generic.go:356] "Generic (PLEG): container finished" podID="932125e5-5bfa-4d5b-82ce-201ca0fa76bd" containerID="a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849" exitCode=0 Jul 09 19:10:19.013192 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.012744 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" Jul 09 19:10:19.013192 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.012788 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" event={"ID":"932125e5-5bfa-4d5b-82ce-201ca0fa76bd","Type":"ContainerDied","Data":"a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849"} Jul 09 19:10:19.013192 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.012832 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fhtjm" event={"ID":"932125e5-5bfa-4d5b-82ce-201ca0fa76bd","Type":"ContainerDied","Data":"06391c8e29ad879646036714fdad407052208728704254b51170ced4f43d1865"} Jul 09 19:10:19.013192 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.012853 2561 scope.go:117] "RemoveContainer" containerID="a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849" Jul 09 19:10:19.014307 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.014277 2561 generic.go:356] "Generic (PLEG): container finished" podID="81764f8b-d8a3-4d10-97ef-345d0dd1c774" containerID="fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06" exitCode=0 Jul 09 19:10:19.014432 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.014363 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rc968" event={"ID":"81764f8b-d8a3-4d10-97ef-345d0dd1c774","Type":"ContainerDied","Data":"fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06"} Jul 09 19:10:19.014432 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.014413 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rc968" event={"ID":"81764f8b-d8a3-4d10-97ef-345d0dd1c774","Type":"ContainerDied","Data":"089fd1188ca6b519bf73cfd2642fe704bb6a94d48290552a039cdd8e6f82be8d"} Jul 09 19:10:19.014432 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.014374 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rc968" Jul 09 19:10:19.032030 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.032010 2561 scope.go:117] "RemoveContainer" containerID="a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849" Jul 09 19:10:19.032361 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:10:19.032339 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849\": container with ID starting with a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849 not found: ID does not exist" containerID="a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849" Jul 09 19:10:19.032466 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.032368 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849"} err="failed to get container status \"a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849\": rpc error: code = NotFound desc = could not find container \"a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849\": container with ID starting with a99ff569f115cd10f5b44af4edf13a4cc75a0a4e40fb3f752f5115be29047849 not found: ID does not exist" Jul 09 19:10:19.032466 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.032387 2561 scope.go:117] "RemoveContainer" containerID="fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06" Jul 09 19:10:19.040985 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.040967 2561 scope.go:117] "RemoveContainer" containerID="fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06" Jul 09 19:10:19.041279 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:10:19.041258 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06\": container with ID starting with fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06 not found: ID does not exist" containerID="fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06" Jul 09 19:10:19.041409 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.041285 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06"} err="failed to get container status \"fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06\": rpc error: code = NotFound desc = could not find container \"fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06\": container with ID starting with fe6cfc29e1d0f7ae26ea019a15ea716559b74f73035a39fa459892de1295ff06 not found: ID does not exist" Jul 09 19:10:19.045258 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.045233 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rc968"] Jul 09 19:10:19.046918 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.046898 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rc968"] Jul 09 19:10:19.057065 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.057043 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fhtjm"] Jul 09 19:10:19.059147 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:19.059123 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fhtjm"] Jul 09 19:10:20.101222 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:20.101190 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81764f8b-d8a3-4d10-97ef-345d0dd1c774" path="/var/lib/kubelet/pods/81764f8b-d8a3-4d10-97ef-345d0dd1c774/volumes" Jul 09 19:10:20.101578 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:20.101566 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932125e5-5bfa-4d5b-82ce-201ca0fa76bd" path="/var/lib/kubelet/pods/932125e5-5bfa-4d5b-82ce-201ca0fa76bd/volumes" Jul 09 19:10:22.314626 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.314585 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:22.335706 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.335667 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78bfd975c5-wzblj"] Jul 09 19:10:22.336057 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.336045 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81764f8b-d8a3-4d10-97ef-345d0dd1c774" containerName="authorino" Jul 09 19:10:22.336101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.336059 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="81764f8b-d8a3-4d10-97ef-345d0dd1c774" containerName="authorino" Jul 09 19:10:22.336101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.336088 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932125e5-5bfa-4d5b-82ce-201ca0fa76bd" containerName="authorino" Jul 09 19:10:22.336101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.336094 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="932125e5-5bfa-4d5b-82ce-201ca0fa76bd" containerName="authorino" Jul 09 19:10:22.336206 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.336148 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="81764f8b-d8a3-4d10-97ef-345d0dd1c774" containerName="authorino" Jul 09 19:10:22.336206 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.336156 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="932125e5-5bfa-4d5b-82ce-201ca0fa76bd" containerName="authorino" Jul 09 19:10:22.419189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.419149 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:22.421835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.421814 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bfd975c5-wzblj"] Jul 09 19:10:22.421941 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.421901 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.424864 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.424840 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 19:10:22.424989 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.424863 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zlkbc\"" Jul 09 19:10:22.424989 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.424867 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 19:10:22.472572 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.472501 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:22.508541 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.508487 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5856874589-blzd9"] Jul 09 19:10:22.508932 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.508910 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-webhook-cert\") pod \"maas-controller-78bfd975c5-wzblj\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.509050 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.509017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx9l7\" (UniqueName: \"kubernetes.io/projected/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-kube-api-access-sx9l7\") pod \"maas-controller-78bfd975c5-wzblj\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.529446 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.529410 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5856874589-blzd9"] Jul 09 19:10:22.529730 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.529543 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.541636 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.541606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:22.610080 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.609990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-webhook-cert\") pod \"maas-controller-5856874589-blzd9\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.610080 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.610040 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-webhook-cert\") pod \"maas-controller-78bfd975c5-wzblj\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.610319 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.610117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svrzn\" (UniqueName: \"kubernetes.io/projected/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-kube-api-access-svrzn\") pod \"maas-controller-5856874589-blzd9\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.610319 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.610139 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx9l7\" (UniqueName: \"kubernetes.io/projected/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-kube-api-access-sx9l7\") pod \"maas-controller-78bfd975c5-wzblj\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.612751 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.612726 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-webhook-cert\") pod \"maas-controller-78bfd975c5-wzblj\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.619232 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.619209 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx9l7\" (UniqueName: \"kubernetes.io/projected/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-kube-api-access-sx9l7\") pod \"maas-controller-78bfd975c5-wzblj\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.711330 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.711289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-webhook-cert\") pod \"maas-controller-5856874589-blzd9\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.711576 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.711407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svrzn\" (UniqueName: \"kubernetes.io/projected/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-kube-api-access-svrzn\") pod \"maas-controller-5856874589-blzd9\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.713963 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.713943 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-webhook-cert\") pod \"maas-controller-5856874589-blzd9\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.719898 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.719878 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svrzn\" (UniqueName: \"kubernetes.io/projected/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-kube-api-access-svrzn\") pod \"maas-controller-5856874589-blzd9\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.733272 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.733245 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:22.840980 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.840939 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:22.887624 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:22.887597 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bfd975c5-wzblj"] Jul 09 19:10:22.890481 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:10:22.890454 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb792ded_d6bb_4f3f_bb07_aadbbb10d042.slice/crio-35c624f6a517ff6fc0954be42367ad322934af8f84cced2bd9991bdd40031b0e WatchSource:0}: Error finding container 35c624f6a517ff6fc0954be42367ad322934af8f84cced2bd9991bdd40031b0e: Status 404 returned error can't find the container with id 35c624f6a517ff6fc0954be42367ad322934af8f84cced2bd9991bdd40031b0e Jul 09 19:10:23.001621 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:23.001597 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5856874589-blzd9"] Jul 09 19:10:23.003574 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:10:23.003544 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb0b993_9dc3_41e3_a434_0bec2f2b9bcf.slice/crio-9b62f3929374993f5b7cbffde65ede45d29a82e54457126daaeb93aa46f5f6f4 WatchSource:0}: Error finding container 9b62f3929374993f5b7cbffde65ede45d29a82e54457126daaeb93aa46f5f6f4: Status 404 returned error can't find the container with id 9b62f3929374993f5b7cbffde65ede45d29a82e54457126daaeb93aa46f5f6f4 Jul 09 19:10:23.035207 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:23.035171 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5856874589-blzd9" event={"ID":"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf","Type":"ContainerStarted","Data":"9b62f3929374993f5b7cbffde65ede45d29a82e54457126daaeb93aa46f5f6f4"} Jul 09 19:10:23.036244 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:23.036211 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfd975c5-wzblj" event={"ID":"cb792ded-d6bb-4f3f-bb07-aadbbb10d042","Type":"ContainerStarted","Data":"35c624f6a517ff6fc0954be42367ad322934af8f84cced2bd9991bdd40031b0e"} Jul 09 19:10:26.056495 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.056387 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5856874589-blzd9" event={"ID":"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf","Type":"ContainerStarted","Data":"4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea"} Jul 09 19:10:26.056931 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.056552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:26.057879 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.057852 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfd975c5-wzblj" event={"ID":"cb792ded-d6bb-4f3f-bb07-aadbbb10d042","Type":"ContainerStarted","Data":"e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75"} Jul 09 19:10:26.058000 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.057986 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:26.075171 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.075124 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5856874589-blzd9" podStartSLOduration=1.324915729 podStartE2EDuration="4.075109286s" totalImagesPullingTime="2.750193557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:10:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:10:26.074182858 +0000 UTC m=+726.525635317" watchObservedRunningTime="2026-07-09 19:10:26.075109286 +0000 UTC m=+726.526561745" Jul 09 19:10:26.099800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.099752 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78bfd975c5-wzblj" podStartSLOduration=1.2464825880000001 podStartE2EDuration="4.099737419s" totalImagesPullingTime="2.853254831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:10:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:10:26.098036549 +0000 UTC m=+726.549489015" watchObservedRunningTime="2026-07-09 19:10:26.099737419 +0000 UTC m=+726.551189881" Jul 09 19:10:26.564618 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.564580 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:26.665652 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.665618 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:26.918912 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:26.918818 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:27.019381 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:27.019345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:32.027944 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.027905 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-855b7d5f54-p9b4c" podUID="a1791fb2-eb37-47c3-b2d5-4c505e74803a" containerName="console" containerID="cri-o://012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f" gracePeriod=15 Jul 09 19:10:32.273460 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.273432 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:10:32.412543 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412434 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-trusted-ca-bundle\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.412543 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412489 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-oauth-serving-cert\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.412800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412564 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-config\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.412800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412622 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-service-ca\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.412800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412660 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ctsk\" (UniqueName: \"kubernetes.io/projected/a1791fb2-eb37-47c3-b2d5-4c505e74803a-kube-api-access-6ctsk\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.412800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412683 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-oauth-config\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.412800 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412705 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-serving-cert\") pod \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\" (UID: \"a1791fb2-eb37-47c3-b2d5-4c505e74803a\") " Jul 09 19:10:32.413048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.412909 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:10:32.413163 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.413124 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:10:32.413163 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.413151 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-config" (OuterVolumeSpecName: "console-config") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:10:32.413330 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.413146 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-trusted-ca-bundle\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:32.413330 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.413290 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-service-ca" (OuterVolumeSpecName: "service-ca") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:10:32.415036 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.415011 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:10:32.415117 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.415043 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:10:32.415151 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.415126 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1791fb2-eb37-47c3-b2d5-4c505e74803a-kube-api-access-6ctsk" (OuterVolumeSpecName: "kube-api-access-6ctsk") pod "a1791fb2-eb37-47c3-b2d5-4c505e74803a" (UID: "a1791fb2-eb37-47c3-b2d5-4c505e74803a"). InnerVolumeSpecName "kube-api-access-6ctsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:10:32.514264 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.514223 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-oauth-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:32.514264 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.514257 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:32.514264 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.514269 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1791fb2-eb37-47c3-b2d5-4c505e74803a-service-ca\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:32.514264 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.514277 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ctsk\" (UniqueName: \"kubernetes.io/projected/a1791fb2-eb37-47c3-b2d5-4c505e74803a-kube-api-access-6ctsk\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:32.514574 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.514286 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-oauth-config\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:32.514574 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:32.514296 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1791fb2-eb37-47c3-b2d5-4c505e74803a-console-serving-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:33.095892 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.095855 2561 generic.go:356] "Generic (PLEG): container finished" podID="a1791fb2-eb37-47c3-b2d5-4c505e74803a" containerID="012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f" exitCode=0 Jul 09 19:10:33.096380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.095903 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855b7d5f54-p9b4c" Jul 09 19:10:33.096380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.095937 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b7d5f54-p9b4c" event={"ID":"a1791fb2-eb37-47c3-b2d5-4c505e74803a","Type":"ContainerDied","Data":"012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f"} Jul 09 19:10:33.096380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.095988 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855b7d5f54-p9b4c" event={"ID":"a1791fb2-eb37-47c3-b2d5-4c505e74803a","Type":"ContainerDied","Data":"7b4eec83644fcecbf80faf2f1d425e76926392ffa3cffa94eae14b6dd7ed09f8"} Jul 09 19:10:33.096380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.096009 2561 scope.go:117] "RemoveContainer" containerID="012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f" Jul 09 19:10:33.105477 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.105462 2561 scope.go:117] "RemoveContainer" containerID="012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f" Jul 09 19:10:33.105754 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:10:33.105737 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f\": container with ID starting with 012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f not found: ID does not exist" containerID="012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f" Jul 09 19:10:33.105803 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.105764 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f"} err="failed to get container status \"012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f\": rpc error: code = NotFound desc = could not find container \"012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f\": container with ID starting with 012c0e08c18ecb27ed008fcf9ca44bf1a2e1e985831afdacb17a6801a891672f not found: ID does not exist" Jul 09 19:10:33.118233 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.118205 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855b7d5f54-p9b4c"] Jul 09 19:10:33.121774 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:33.121749 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-855b7d5f54-p9b4c"] Jul 09 19:10:34.101045 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:34.101014 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1791fb2-eb37-47c3-b2d5-4c505e74803a" path="/var/lib/kubelet/pods/a1791fb2-eb37-47c3-b2d5-4c505e74803a/volumes" Jul 09 19:10:37.068938 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.068905 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:37.069324 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.068982 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:10:37.129661 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.129628 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bfd975c5-wzblj"] Jul 09 19:10:37.129885 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.129855 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78bfd975c5-wzblj" podUID="cb792ded-d6bb-4f3f-bb07-aadbbb10d042" containerName="manager" containerID="cri-o://e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75" gracePeriod=30 Jul 09 19:10:37.377471 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.377445 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:37.463454 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.463410 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx9l7\" (UniqueName: \"kubernetes.io/projected/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-kube-api-access-sx9l7\") pod \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " Jul 09 19:10:37.463675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.463547 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-webhook-cert\") pod \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\" (UID: \"cb792ded-d6bb-4f3f-bb07-aadbbb10d042\") " Jul 09 19:10:37.465924 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.465893 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cb792ded-d6bb-4f3f-bb07-aadbbb10d042" (UID: "cb792ded-d6bb-4f3f-bb07-aadbbb10d042"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:10:37.465997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.465900 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-kube-api-access-sx9l7" (OuterVolumeSpecName: "kube-api-access-sx9l7") pod "cb792ded-d6bb-4f3f-bb07-aadbbb10d042" (UID: "cb792ded-d6bb-4f3f-bb07-aadbbb10d042"). InnerVolumeSpecName "kube-api-access-sx9l7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:10:37.564827 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.564786 2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-webhook-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:37.564827 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:37.564821 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx9l7\" (UniqueName: \"kubernetes.io/projected/cb792ded-d6bb-4f3f-bb07-aadbbb10d042-kube-api-access-sx9l7\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:10:38.123089 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.123053 2561 generic.go:356] "Generic (PLEG): container finished" podID="cb792ded-d6bb-4f3f-bb07-aadbbb10d042" containerID="e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75" exitCode=0 Jul 09 19:10:38.123805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.123102 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfd975c5-wzblj" Jul 09 19:10:38.123805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.123137 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfd975c5-wzblj" event={"ID":"cb792ded-d6bb-4f3f-bb07-aadbbb10d042","Type":"ContainerDied","Data":"e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75"} Jul 09 19:10:38.123805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.123192 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfd975c5-wzblj" event={"ID":"cb792ded-d6bb-4f3f-bb07-aadbbb10d042","Type":"ContainerDied","Data":"35c624f6a517ff6fc0954be42367ad322934af8f84cced2bd9991bdd40031b0e"} Jul 09 19:10:38.123805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.123216 2561 scope.go:117] "RemoveContainer" containerID="e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75" Jul 09 19:10:38.132075 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.131911 2561 scope.go:117] "RemoveContainer" containerID="e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75" Jul 09 19:10:38.132241 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:10:38.132219 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75\": container with ID starting with e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75 not found: ID does not exist" containerID="e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75" Jul 09 19:10:38.132314 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.132254 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75"} err="failed to get container status \"e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75\": rpc error: code = NotFound desc = could not find container \"e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75\": container with ID starting with e5df25b435a1f1a5f9ea9626ffa26ece41ba9339d46340a635da61bcfddd1b75 not found: ID does not exist" Jul 09 19:10:38.140099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.140075 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bfd975c5-wzblj"] Jul 09 19:10:38.141824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:38.141807 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78bfd975c5-wzblj"] Jul 09 19:10:40.106003 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:40.105966 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb792ded-d6bb-4f3f-bb07-aadbbb10d042" path="/var/lib/kubelet/pods/cb792ded-d6bb-4f3f-bb07-aadbbb10d042/volumes" Jul 09 19:10:59.526855 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:10:59.526807 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:00.060287 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.060249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:00.129279 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129246 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8486c54598-2xkbc"] Jul 09 19:11:00.129666 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129652 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb792ded-d6bb-4f3f-bb07-aadbbb10d042" containerName="manager" Jul 09 19:11:00.129725 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129667 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb792ded-d6bb-4f3f-bb07-aadbbb10d042" containerName="manager" Jul 09 19:11:00.129725 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129680 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1791fb2-eb37-47c3-b2d5-4c505e74803a" containerName="console" Jul 09 19:11:00.129725 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129686 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1791fb2-eb37-47c3-b2d5-4c505e74803a" containerName="console" Jul 09 19:11:00.129818 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129741 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1791fb2-eb37-47c3-b2d5-4c505e74803a" containerName="console" Jul 09 19:11:00.129818 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.129751 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb792ded-d6bb-4f3f-bb07-aadbbb10d042" containerName="manager" Jul 09 19:11:00.137089 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.137064 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.140896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.140870 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 19:11:00.140896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.140870 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9pdt2\"" Jul 09 19:11:00.142090 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.142064 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8486c54598-2xkbc"] Jul 09 19:11:00.169675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.169647 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52905b97-b7e2-40b6-901f-f8c45c2c8414-maas-api-tls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.169675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.169679 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52905b97-b7e2-40b6-901f-f8c45c2c8414-openshift-service-ca\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.169921 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.169751 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl9ls\" (UniqueName: \"kubernetes.io/projected/52905b97-b7e2-40b6-901f-f8c45c2c8414-kube-api-access-zl9ls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.270437 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.270400 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52905b97-b7e2-40b6-901f-f8c45c2c8414-maas-api-tls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.270629 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.270447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52905b97-b7e2-40b6-901f-f8c45c2c8414-openshift-service-ca\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.270629 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.270538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl9ls\" (UniqueName: \"kubernetes.io/projected/52905b97-b7e2-40b6-901f-f8c45c2c8414-kube-api-access-zl9ls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.270629 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:11:00.270586 2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 09 19:11:00.270815 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:11:00.270667 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52905b97-b7e2-40b6-901f-f8c45c2c8414-maas-api-tls podName:52905b97-b7e2-40b6-901f-f8c45c2c8414 nodeName:}" failed. No retries permitted until 2026-07-09 19:11:00.770643841 +0000 UTC m=+761.222096282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/52905b97-b7e2-40b6-901f-f8c45c2c8414-maas-api-tls") pod "maas-api-8486c54598-2xkbc" (UID: "52905b97-b7e2-40b6-901f-f8c45c2c8414") : secret "maas-api-serving-cert" not found Jul 09 19:11:00.271194 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.271175 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52905b97-b7e2-40b6-901f-f8c45c2c8414-openshift-service-ca\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.282014 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.281988 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl9ls\" (UniqueName: \"kubernetes.io/projected/52905b97-b7e2-40b6-901f-f8c45c2c8414-kube-api-access-zl9ls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.488002 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.487964 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:00.587401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.587363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:00.678849 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.678809 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-cjnxj"] Jul 09 19:11:00.684170 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.684150 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.688870 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.688850 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 19:11:00.689062 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.689041 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cqk8d\"" Jul 09 19:11:00.695664 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.695635 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-cjnxj"] Jul 09 19:11:00.774925 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.774829 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp26v\" (UniqueName: \"kubernetes.io/projected/8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1-kube-api-access-gp26v\") pod \"payload-pre-processing-7964457696-cjnxj\" (UID: \"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1\") " pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.775099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.774923 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-cjnxj\" (UID: \"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1\") " pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.775099 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.775054 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52905b97-b7e2-40b6-901f-f8c45c2c8414-maas-api-tls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.777638 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.777614 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52905b97-b7e2-40b6-901f-f8c45c2c8414-maas-api-tls\") pod \"maas-api-8486c54598-2xkbc\" (UID: \"52905b97-b7e2-40b6-901f-f8c45c2c8414\") " pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:00.875838 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.875805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-cjnxj\" (UID: \"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1\") " pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.876020 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.875878 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp26v\" (UniqueName: \"kubernetes.io/projected/8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1-kube-api-access-gp26v\") pod \"payload-pre-processing-7964457696-cjnxj\" (UID: \"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1\") " pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.876542 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.876494 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-cjnxj\" (UID: \"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1\") " pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.884425 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.884398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp26v\" (UniqueName: \"kubernetes.io/projected/8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1-kube-api-access-gp26v\") pod \"payload-pre-processing-7964457696-cjnxj\" (UID: \"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1\") " pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:00.996170 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:00.996123 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:01.049363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:01.049261 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:01.150454 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:01.150408 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-cjnxj"] Jul 09 19:11:01.152370 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:11:01.152337 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee7cb9f_1301_4302_a3f1_6f4eceaab2d1.slice/crio-46d843e7d31c6318dc25ade0dbe2e56c5fc3ac25c234484d4083fd5cd7c3b78d WatchSource:0}: Error finding container 46d843e7d31c6318dc25ade0dbe2e56c5fc3ac25c234484d4083fd5cd7c3b78d: Status 404 returned error can't find the container with id 46d843e7d31c6318dc25ade0dbe2e56c5fc3ac25c234484d4083fd5cd7c3b78d Jul 09 19:11:01.210312 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:01.210286 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8486c54598-2xkbc"] Jul 09 19:11:01.212044 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:11:01.212014 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52905b97_b7e2_40b6_901f_f8c45c2c8414.slice/crio-42905c799b750b1ef82d283d280da905f3d0b11111c25981c5c2c765e7e68050 WatchSource:0}: Error finding container 42905c799b750b1ef82d283d280da905f3d0b11111c25981c5c2c765e7e68050: Status 404 returned error can't find the container with id 42905c799b750b1ef82d283d280da905f3d0b11111c25981c5c2c765e7e68050 Jul 09 19:11:01.237062 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:01.237020 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" event={"ID":"52905b97-b7e2-40b6-901f-f8c45c2c8414","Type":"ContainerStarted","Data":"42905c799b750b1ef82d283d280da905f3d0b11111c25981c5c2c765e7e68050"} Jul 09 19:11:01.238054 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:01.238029 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" event={"ID":"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1","Type":"ContainerStarted","Data":"46d843e7d31c6318dc25ade0dbe2e56c5fc3ac25c234484d4083fd5cd7c3b78d"} Jul 09 19:11:03.249769 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:03.249728 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" event={"ID":"8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1","Type":"ContainerStarted","Data":"659dbdda4ce2cb4cf97de1a21cb1cfacb8f364534e4ed9b082ab08537bd95c5c"} Jul 09 19:11:03.250244 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:03.249878 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:03.267215 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:03.267122 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" podStartSLOduration=1.774742862 podStartE2EDuration="3.267104391s" totalImagesPullingTime="1.492361529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:03.266344684 +0000 UTC m=+763.717797145" watchObservedRunningTime="2026-07-09 19:11:03.267104391 +0000 UTC m=+763.718556843" Jul 09 19:11:04.256031 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:04.255937 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" event={"ID":"52905b97-b7e2-40b6-901f-f8c45c2c8414","Type":"ContainerStarted","Data":"fb104e1a86ba6ca0816c0d84e4df5a6f39c2fe1c2077d9a89c536844c4ee19bf"} Jul 09 19:11:04.256464 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:04.256227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:04.275017 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:04.274969 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" podStartSLOduration=1.49850981 podStartE2EDuration="4.274953821s" totalImagesPullingTime="2.776444011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:04.272798087 +0000 UTC m=+764.724250558" watchObservedRunningTime="2026-07-09 19:11:04.274953821 +0000 UTC m=+764.726406280" Jul 09 19:11:10.265826 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:10.265799 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8486c54598-2xkbc" Jul 09 19:11:11.247619 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:11.247563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:11.348154 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:11.348095 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:14.258987 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:14.258959 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-cjnxj" Jul 09 19:11:23.955221 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.955177 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj"] Jul 09 19:11:23.960360 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.960338 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:23.964336 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.964308 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 19:11:23.964582 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.964354 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 19:11:23.964582 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.964405 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 19:11:23.964582 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.964475 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5jbdw\"" Jul 09 19:11:23.967536 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.967476 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj"] Jul 09 19:11:23.981136 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:23.981085 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:24.115240 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.115201 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.115443 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.115280 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.115443 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.115316 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rtxv\" (UniqueName: \"kubernetes.io/projected/5da3c67c-b571-4fd4-a48b-364d8d9e7166-kube-api-access-9rtxv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.115443 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.115345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.115443 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.115379 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da3c67c-b571-4fd4-a48b-364d8d9e7166-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.115443 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.115405 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.216789 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.216690 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.216975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.216800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.216975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.216883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.217113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.216949 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rtxv\" (UniqueName: \"kubernetes.io/projected/5da3c67c-b571-4fd4-a48b-364d8d9e7166-kube-api-access-9rtxv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.217113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.217042 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.217113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.217074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da3c67c-b571-4fd4-a48b-364d8d9e7166-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.217244 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.217147 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.217244 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.217173 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.217383 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.217363 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.219296 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.219266 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5da3c67c-b571-4fd4-a48b-364d8d9e7166-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.219540 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.219502 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5da3c67c-b571-4fd4-a48b-364d8d9e7166-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.225226 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.225200 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rtxv\" (UniqueName: \"kubernetes.io/projected/5da3c67c-b571-4fd4-a48b-364d8d9e7166-kube-api-access-9rtxv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj\" (UID: \"5da3c67c-b571-4fd4-a48b-364d8d9e7166\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.273236 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.273194 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:24.412952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:24.412925 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj"] Jul 09 19:11:24.414888 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:11:24.414859 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da3c67c_b571_4fd4_a48b_364d8d9e7166.slice/crio-14c6ba65e1fc47c66ca75f8ba85ab18a6a6cd56fce4ad48548f87e620013a75f WatchSource:0}: Error finding container 14c6ba65e1fc47c66ca75f8ba85ab18a6a6cd56fce4ad48548f87e620013a75f: Status 404 returned error can't find the container with id 14c6ba65e1fc47c66ca75f8ba85ab18a6a6cd56fce4ad48548f87e620013a75f Jul 09 19:11:25.360381 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:25.360338 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" event={"ID":"5da3c67c-b571-4fd4-a48b-364d8d9e7166","Type":"ContainerStarted","Data":"14c6ba65e1fc47c66ca75f8ba85ab18a6a6cd56fce4ad48548f87e620013a75f"} Jul 09 19:11:26.367385 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:26.367342 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" event={"ID":"5da3c67c-b571-4fd4-a48b-364d8d9e7166","Type":"ContainerStarted","Data":"7096706fd25cca1503022fa9d08237519b01d30a93ab9a1834e839f8666df61b"} Jul 09 19:11:29.227216 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:29.227172 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:34.274274 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:34.274235 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:34.274754 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:34.274304 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:34.287247 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:34.287213 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:34.307034 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:34.306970 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" podStartSLOduration=10.19263534 podStartE2EDuration="11.306951102s" totalImagesPullingTime="1.114315762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:26.388683988 +0000 UTC m=+786.840136472" watchObservedRunningTime="2026-07-09 19:11:34.306951102 +0000 UTC m=+794.758403583" Jul 09 19:11:34.420918 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:34.420884 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj" Jul 09 19:11:35.354171 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:35.354131 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:35.454809 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:35.454773 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:42.095676 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:42.095571 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.367723 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.367689 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8"] Jul 09 19:11:55.370647 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.370629 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.373243 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.373220 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 19:11:55.380692 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.380668 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8"] Jul 09 19:11:55.390732 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.390702 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.510673 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.510642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.510673 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.510675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.510905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.510710 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.510905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.510817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9656346-57ae-4671-9950-f54388b0f2aa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.510905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.510860 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjdxw\" (UniqueName: \"kubernetes.io/projected/b9656346-57ae-4671-9950-f54388b0f2aa-kube-api-access-zjdxw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.510905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.510883 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.611626 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.611578 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjdxw\" (UniqueName: \"kubernetes.io/projected/b9656346-57ae-4671-9950-f54388b0f2aa-kube-api-access-zjdxw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.611626 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.611630 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.611896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.611686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.611896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.611709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.611896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.611763 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.611896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.611843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9656346-57ae-4671-9950-f54388b0f2aa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.612113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.612049 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.612172 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.612109 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.612172 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.612137 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.614170 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.614140 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9656346-57ae-4671-9950-f54388b0f2aa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.614450 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.614433 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9656346-57ae-4671-9950-f54388b0f2aa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.629705 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.629644 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjdxw\" (UniqueName: \"kubernetes.io/projected/b9656346-57ae-4671-9950-f54388b0f2aa-kube-api-access-zjdxw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-prvk8\" (UID: \"b9656346-57ae-4671-9950-f54388b0f2aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.681724 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.681682 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:11:55.820568 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:55.820541 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8"] Jul 09 19:11:56.512915 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:56.512877 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" event={"ID":"b9656346-57ae-4671-9950-f54388b0f2aa","Type":"ContainerStarted","Data":"70068cf58727cc703e0acb328769130c73d0f61ff41b2020264043b81afdfb44"} Jul 09 19:11:56.513319 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:56.512937 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" event={"ID":"b9656346-57ae-4671-9950-f54388b0f2aa","Type":"ContainerStarted","Data":"33ea2cb4961480dcc8cffcf7b4aa0a7e9b8a7a894f9485ffc2f4e0dc93d95b11"} Jul 09 19:11:56.534696 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:11:56.534645 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" podStartSLOduration=1.304011138 podStartE2EDuration="1.534623207s" totalImagesPullingTime="230.612069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:56.532164656 +0000 UTC m=+816.983617117" watchObservedRunningTime="2026-07-09 19:11:56.534623207 +0000 UTC m=+816.986075666" Jul 09 19:12:01.988268 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:01.988217 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:05.682429 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:05.682389 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:12:05.682881 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:05.682560 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:12:05.694926 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:05.694890 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:12:06.456297 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.456263 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr"] Jul 09 19:12:06.462947 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.462921 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.465797 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.465774 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 19:12:06.468499 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.468473 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr"] Jul 09 19:12:06.480358 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.480328 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:06.572350 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.572317 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-prvk8" Jul 09 19:12:06.621112 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.621084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.621277 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.621126 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.621277 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.621238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.621362 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.621313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.621362 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.621347 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt58w\" (UniqueName: \"kubernetes.io/projected/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-kube-api-access-zt58w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.621438 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.621417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.722461 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.722461 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722490 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt58w\" (UniqueName: \"kubernetes.io/projected/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-kube-api-access-zt58w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722876 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722978 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.723043 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.722996 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.725137 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.725116 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.725400 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.725378 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.731898 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.731875 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt58w\" (UniqueName: \"kubernetes.io/projected/3e668f44-e6ea-4b3d-b6ac-70cc0177b489-kube-api-access-zt58w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzfkr\" (UID: \"3e668f44-e6ea-4b3d-b6ac-70cc0177b489\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.775713 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.775680 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:06.916562 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:06.916434 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr"] Jul 09 19:12:06.919500 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:12:06.919472 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e668f44_e6ea_4b3d_b6ac_70cc0177b489.slice/crio-e5772b879e7f9b0c577afb8a07527f95f46b0970da116b0a0200b7e1c1cbb791 WatchSource:0}: Error finding container e5772b879e7f9b0c577afb8a07527f95f46b0970da116b0a0200b7e1c1cbb791: Status 404 returned error can't find the container with id e5772b879e7f9b0c577afb8a07527f95f46b0970da116b0a0200b7e1c1cbb791 Jul 09 19:12:07.568158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:07.568125 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" event={"ID":"3e668f44-e6ea-4b3d-b6ac-70cc0177b489","Type":"ContainerStarted","Data":"18cf644a87473d74b0441b31bf72ecddb8198aef8d7aadf5903b539276ab0e48"} Jul 09 19:12:07.568333 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:07.568182 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" event={"ID":"3e668f44-e6ea-4b3d-b6ac-70cc0177b489","Type":"ContainerStarted","Data":"e5772b879e7f9b0c577afb8a07527f95f46b0970da116b0a0200b7e1c1cbb791"} Jul 09 19:12:07.587198 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:07.587139 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" podStartSLOduration=1.355660382 podStartE2EDuration="1.58712372s" totalImagesPullingTime="231.463338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:12:07.585879813 +0000 UTC m=+828.037332272" watchObservedRunningTime="2026-07-09 19:12:07.58712372 +0000 UTC m=+828.038576180" Jul 09 19:12:16.775851 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:16.775817 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:16.776278 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:16.775880 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:16.788453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:16.788429 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:17.635599 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:17.635573 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzfkr" Jul 09 19:12:19.058483 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.058439 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9"] Jul 09 19:12:19.062017 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.062000 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.064700 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.064673 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 19:12:19.073320 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.073298 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9"] Jul 09 19:12:19.089260 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.089229 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:19.142654 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.142621 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.142811 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.142684 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08558249-01ac-45f9-87b3-83b82fef61f9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.142811 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.142711 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.142811 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.142755 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.142811 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.142804 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lczlp\" (UniqueName: \"kubernetes.io/projected/08558249-01ac-45f9-87b3-83b82fef61f9-kube-api-access-lczlp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.142947 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.142828 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.243989 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.243927 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08558249-01ac-45f9-87b3-83b82fef61f9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244194 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244011 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244194 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244080 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244194 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244133 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lczlp\" (UniqueName: \"kubernetes.io/projected/08558249-01ac-45f9-87b3-83b82fef61f9-kube-api-access-lczlp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244194 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244168 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244394 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244626 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244595 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244626 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244614 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.244909 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.244702 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.246451 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.246421 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08558249-01ac-45f9-87b3-83b82fef61f9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.246773 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.246753 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08558249-01ac-45f9-87b3-83b82fef61f9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.252656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.252637 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lczlp\" (UniqueName: \"kubernetes.io/projected/08558249-01ac-45f9-87b3-83b82fef61f9-kube-api-access-lczlp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9\" (UID: \"08558249-01ac-45f9-87b3-83b82fef61f9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.373002 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.372909 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:19.509271 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.509237 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9"] Jul 09 19:12:19.512056 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:12:19.512025 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08558249_01ac_45f9_87b3_83b82fef61f9.slice/crio-b1f2bc5a2d2b2567420df5b298ac0c41c8c872e97ea480f67895e13a63a522f2 WatchSource:0}: Error finding container b1f2bc5a2d2b2567420df5b298ac0c41c8c872e97ea480f67895e13a63a522f2: Status 404 returned error can't find the container with id b1f2bc5a2d2b2567420df5b298ac0c41c8c872e97ea480f67895e13a63a522f2 Jul 09 19:12:19.635151 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:19.635065 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" event={"ID":"08558249-01ac-45f9-87b3-83b82fef61f9","Type":"ContainerStarted","Data":"b1f2bc5a2d2b2567420df5b298ac0c41c8c872e97ea480f67895e13a63a522f2"} Jul 09 19:12:20.641787 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:20.641746 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" event={"ID":"08558249-01ac-45f9-87b3-83b82fef61f9","Type":"ContainerStarted","Data":"689c6b47b7d3a8d759b50b2560d5ac644f0df268a49d8dc2f9ef73876dbe4a9a"} Jul 09 19:12:20.659906 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:20.659852 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" podStartSLOduration=1.425362354 podStartE2EDuration="1.659836822s" totalImagesPullingTime="234.474468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:12:20.659130976 +0000 UTC m=+841.110583457" watchObservedRunningTime="2026-07-09 19:12:20.659836822 +0000 UTC m=+841.111289282" Jul 09 19:12:29.373242 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:29.373195 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:29.373657 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:29.373278 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:29.385590 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:29.385573 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:29.695285 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:29.695205 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9" Jul 09 19:12:43.562277 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:12:43.562237 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:03.821529 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:03.821473 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:05.700368 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:05.700329 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:09.385060 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:09.385025 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:09.482386 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:09.482353 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:20.045494 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:20.045411 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:13:20.052071 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:20.052047 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:13:54.754651 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:13:54.754614 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:48.648957 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:48.648869 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:48.750233 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:48.750183 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5856874589-blzd9"] Jul 09 19:14:48.750449 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:48.750421 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5856874589-blzd9" podUID="7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" containerName="manager" containerID="cri-o://4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea" gracePeriod=30 Jul 09 19:14:49.002922 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.002894 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:14:49.110627 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.110582 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svrzn\" (UniqueName: \"kubernetes.io/projected/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-kube-api-access-svrzn\") pod \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " Jul 09 19:14:49.110824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.110705 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-webhook-cert\") pod \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\" (UID: \"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf\") " Jul 09 19:14:49.112918 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.112886 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" (UID: "7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:14:49.112918 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.112891 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-kube-api-access-svrzn" (OuterVolumeSpecName: "kube-api-access-svrzn") pod "7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" (UID: "7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf"). InnerVolumeSpecName "kube-api-access-svrzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:14:49.212038 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.211913 2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-webhook-cert\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:14:49.212038 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.211963 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svrzn\" (UniqueName: \"kubernetes.io/projected/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf-kube-api-access-svrzn\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:14:49.389483 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.389447 2561 generic.go:356] "Generic (PLEG): container finished" podID="7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" containerID="4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea" exitCode=0 Jul 09 19:14:49.389680 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.389526 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5856874589-blzd9" Jul 09 19:14:49.389680 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.389535 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5856874589-blzd9" event={"ID":"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf","Type":"ContainerDied","Data":"4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea"} Jul 09 19:14:49.389680 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.389590 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5856874589-blzd9" event={"ID":"7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf","Type":"ContainerDied","Data":"9b62f3929374993f5b7cbffde65ede45d29a82e54457126daaeb93aa46f5f6f4"} Jul 09 19:14:49.389680 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.389616 2561 scope.go:117] "RemoveContainer" containerID="4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea" Jul 09 19:14:49.399116 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.399096 2561 scope.go:117] "RemoveContainer" containerID="4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea" Jul 09 19:14:49.399416 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:14:49.399395 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea\": container with ID starting with 4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea not found: ID does not exist" containerID="4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea" Jul 09 19:14:49.399460 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.399428 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea"} err="failed to get container status \"4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea\": rpc error: code = NotFound desc = could not find container \"4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea\": container with ID starting with 4f615bdc517d720758d06fc0ad226de3dc435df09e6b4b4327c74c235baf94ea not found: ID does not exist" Jul 09 19:14:49.411296 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.411234 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5856874589-blzd9"] Jul 09 19:14:49.413643 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:49.413614 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5856874589-blzd9"] Jul 09 19:14:50.063907 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.063873 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:50.101149 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.101112 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" path="/var/lib/kubelet/pods/7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf/volumes" Jul 09 19:14:50.176979 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.176942 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5856874589-nv5m5"] Jul 09 19:14:50.177358 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.177346 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" containerName="manager" Jul 09 19:14:50.177402 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.177361 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" containerName="manager" Jul 09 19:14:50.177439 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.177430 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb0b993-9dc3-41e3-a434-0bec2f2b9bcf" containerName="manager" Jul 09 19:14:50.180924 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.180879 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.184874 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.184852 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zlkbc\"" Jul 09 19:14:50.185017 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.184853 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 19:14:50.185017 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.184853 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 19:14:50.188993 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.188970 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5856874589-nv5m5"] Jul 09 19:14:50.222438 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.222398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1680d844-21aa-4977-9e0c-86e88dc05612-webhook-cert\") pod \"maas-controller-5856874589-nv5m5\" (UID: \"1680d844-21aa-4977-9e0c-86e88dc05612\") " pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.222627 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.222466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd6rh\" (UniqueName: \"kubernetes.io/projected/1680d844-21aa-4977-9e0c-86e88dc05612-kube-api-access-wd6rh\") pod \"maas-controller-5856874589-nv5m5\" (UID: \"1680d844-21aa-4977-9e0c-86e88dc05612\") " pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.323353 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.323315 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd6rh\" (UniqueName: \"kubernetes.io/projected/1680d844-21aa-4977-9e0c-86e88dc05612-kube-api-access-wd6rh\") pod \"maas-controller-5856874589-nv5m5\" (UID: \"1680d844-21aa-4977-9e0c-86e88dc05612\") " pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.323576 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.323442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1680d844-21aa-4977-9e0c-86e88dc05612-webhook-cert\") pod \"maas-controller-5856874589-nv5m5\" (UID: \"1680d844-21aa-4977-9e0c-86e88dc05612\") " pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.326036 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.326016 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1680d844-21aa-4977-9e0c-86e88dc05612-webhook-cert\") pod \"maas-controller-5856874589-nv5m5\" (UID: \"1680d844-21aa-4977-9e0c-86e88dc05612\") " pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.331773 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.331746 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd6rh\" (UniqueName: \"kubernetes.io/projected/1680d844-21aa-4977-9e0c-86e88dc05612-kube-api-access-wd6rh\") pod \"maas-controller-5856874589-nv5m5\" (UID: \"1680d844-21aa-4977-9e0c-86e88dc05612\") " pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.492551 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.492499 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:50.637027 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.637003 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5856874589-nv5m5"] Jul 09 19:14:50.639072 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:14:50.639033 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1680d844_21aa_4977_9e0c_86e88dc05612.slice/crio-36b570a896a6f896ff20ecd6bd0d45a336d9edb13416e4779530d1542e938c0e WatchSource:0}: Error finding container 36b570a896a6f896ff20ecd6bd0d45a336d9edb13416e4779530d1542e938c0e: Status 404 returned error can't find the container with id 36b570a896a6f896ff20ecd6bd0d45a336d9edb13416e4779530d1542e938c0e Jul 09 19:14:50.640388 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:50.640369 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:14:51.402988 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:51.402946 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5856874589-nv5m5" event={"ID":"1680d844-21aa-4977-9e0c-86e88dc05612","Type":"ContainerStarted","Data":"e7e60e8aada9e4e4bedf10bb2d0afcff3c2d2492e6b837c81ef1ebcb14af3c17"} Jul 09 19:14:51.403409 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:51.403002 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5856874589-nv5m5" event={"ID":"1680d844-21aa-4977-9e0c-86e88dc05612","Type":"ContainerStarted","Data":"36b570a896a6f896ff20ecd6bd0d45a336d9edb13416e4779530d1542e938c0e"} Jul 09 19:14:51.403409 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:51.403037 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:14:51.422115 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:14:51.422064 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5856874589-nv5m5" podStartSLOduration=0.859836827 podStartE2EDuration="1.422049233s" totalImagesPullingTime="562.212406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:14:51.420896373 +0000 UTC m=+991.872348835" watchObservedRunningTime="2026-07-09 19:14:51.422049233 +0000 UTC m=+991.873501712" Jul 09 19:15:02.413401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:02.413371 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5856874589-nv5m5" Jul 09 19:15:30.703967 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:30.703915 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:30.803137 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:30.803102 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:30.935197 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:30.935159 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:36.660756 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:36.660723 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:36.768070 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:36.768034 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:36.867546 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:36.867486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:36.990805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:36.990772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:42.930182 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:42.930149 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:42.993615 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:42.993584 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:55.285004 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:15:55.284962 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:07.386812 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:16:07.386772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:07.447814 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:16:07.447779 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:20.080439 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:18:20.080410 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:18:20.089949 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:18:20.089926 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:18:27.888803 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:18:27.888763 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:18.419415 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:19:18.419298 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:23:20.119444 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:23:20.119413 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:23:20.124597 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:23:20.124576 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:24:15.615975 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:24:15.615922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:17.068982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:24:17.068947 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:43.856706 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:24:43.856659 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:31.454379 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:31.454341 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:38.822996 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:38.822957 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:39.864484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:39.864450 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:41.628104 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:41.628021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:41.728161 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:41.728132 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:47.112786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:47.112754 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:52.193217 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:52.193187 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:55.031730 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:55.031693 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:55.134421 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:55.134388 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:55.505459 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:26:55.505428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:01.817274 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:01.817242 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:01.917437 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:01.917403 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:02.039730 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:02.039684 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:02.115814 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:02.115735 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.196554 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:03.196489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.298284 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:03.298242 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.416535 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:03.416477 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.715545 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:03.715486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.813075 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:03.813043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.915025 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:03.914991 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:06.009088 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:06.009052 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:08.368140 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:08.368085 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:08.437872 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:08.437838 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:08.647574 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:08.647466 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:08.712927 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:08.712878 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:10.463458 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:10.463426 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:14.951973 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:14.951933 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:15.023840 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:15.023809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:15.196608 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:15.196571 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:15.299039 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:15.299003 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:16.314794 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:16.314757 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:16.413193 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:16.413157 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:16.518453 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:16.518418 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:16.755674 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:16.755643 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:17.055983 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:17.055888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:17.156122 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:17.156078 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:18.578877 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:18.578845 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:21.397035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:21.396997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:21.844013 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:21.843980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:27.504474 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:27.504438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:27.611293 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:27.611257 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:29.958896 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:29.958862 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:33.545122 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:33.545084 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:33.645426 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:33.645396 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:33.769063 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:33.769023 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:33.869900 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:33.869816 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:34.965240 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:34.965203 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:35.053902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:35.053868 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:35.202716 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:35.202688 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:35.298131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:35.298100 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:35.643982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:35.643883 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:35.744594 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:35.744550 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:37.094046 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:37.093997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:39.318902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:39.318843 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:39.533638 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:39.533597 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:39.634380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:39.634297 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:39.793489 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:39.793460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:41.584802 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:41.584768 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:45.859620 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:45.859585 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:45.961146 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:45.961112 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:46.349795 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:46.349759 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:46.449897 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:46.449862 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:46.552371 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:46.552336 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:47.968269 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:47.968231 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:54.420452 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:54.420423 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:54.524355 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:54.524324 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:54.623534 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:54.623479 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:54.721894 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:54.721811 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:55.150544 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:27:55.150497 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:20.151069 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:20.151037 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:28:20.156290 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:20.156269 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:28:21.118924 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:21.118887 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:27.260238 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:27.260200 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:27.359958 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:27.359925 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:43.144067 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:43.144014 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:48.960806 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:48.960768 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:49.071431 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:49.071394 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:49.199755 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:49.199714 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:28:59.681368 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:28:59.681333 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:00.459179 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:00.459147 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:06.345132 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:06.345100 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:06.448563 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:06.448520 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:06.540049 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:06.540010 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:06.640189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:06.640109 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:07.403808 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:07.403778 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:07.521382 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:07.521319 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:07.699397 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:07.699315 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:09.169803 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:09.169772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:15.379936 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:15.379901 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:15.452044 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:15.452014 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:16.063904 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:16.063868 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:21.964883 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:21.964844 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:23.338203 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:23.338164 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:25.318039 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:25.318000 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:25.405190 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:25.405155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:25.476048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:25.475976 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:25.665154 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:25.665049 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:25.969986 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:25.969893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:26.071902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:26.071866 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:29.804674 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:29.804637 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:30.464459 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:30.464428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:30.553069 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:30.553002 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:30.877422 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:30.877387 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:36.653059 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:36.652997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:36.726691 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:36.726661 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:37.043878 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:37.043848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:37.466066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:37.465975 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:37.538690 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:37.538640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:43.372855 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:43.372819 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:43.473610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:43.473577 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:43.635330 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:43.635243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:43.845481 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:43.845452 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:43.976893 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:43.976764 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:44.079892 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:44.079863 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:44.187415 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:44.187324 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:50.205255 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:50.205216 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:50.301682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:50.301644 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:53.444311 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:53.444271 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:53.548251 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:53.548210 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:53.644126 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:53.644095 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:59.562380 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:59.562348 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:29:59.624599 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:29:59.624564 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:06.365610 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:06.365560 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:06.667012 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:06.666918 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:06.775276 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:06.775237 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:09.607007 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:09.606968 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:09.700726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:09.700696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:09.790785 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:09.790750 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:09.891262 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:09.891174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:10.213531 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:10.213431 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:10.318810 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:10.318750 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:12.180734 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:12.180696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:12.280539 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:12.280494 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:14.113647 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:14.113611 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:25.296027 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:25.295987 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:32.030222 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:32.030189 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:32.128167 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:32.128129 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:46.751564 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:46.751490 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:47.164666 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:47.164579 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:52.742169 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:52.742130 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:56.336078 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:56.336043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:56.436470 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:56.436417 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:30:57.781038 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:30:57.781002 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:34.797525 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:34.797459 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:35.003038 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:35.003005 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:40.593162 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:40.593118 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:40.664883 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:40.664847 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:40.904130 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:40.904039 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:41.146673 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:41.146638 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:42.183924 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:42.183888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:42.285737 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:42.285704 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:42.385952 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:42.385921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:43.120955 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:43.120922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:43.226577 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:43.226540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:43.361545 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:43.361473 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:44.721917 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:44.721877 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:50.749983 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:50.749946 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:50.824147 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:50.824116 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:51.457911 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:51.457873 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:57.535897 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:57.535864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:57.637581 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:57.637547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:57.735328 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:57.735294 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:57.838455 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:57.838365 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:31:58.239715 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:31:58.239687 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:06.253003 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:06.252965 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:08.270484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:08.270445 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:08.617488 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:08.617400 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:29.563205 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:29.563168 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:35.548199 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:35.548164 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:35.648704 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:35.648661 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:35.883300 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:35.883212 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:36.188302 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:36.188215 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:36.517815 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:36.517783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:39.493204 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:39.493159 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:39.580494 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:39.580451 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:39.678689 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:39.678656 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:40.063660 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:40.063624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:40.161416 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:40.161374 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:40.405560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:40.405460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:43.964415 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:43.964332 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:44.294690 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:44.294655 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:44.595007 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:44.594922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:44.940066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:44.939974 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:47.528876 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:47.528841 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:47.622726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:47.622694 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:47.776914 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:47.776858 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:47.882408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:47.882306 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:48.308045 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:48.308014 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:48.405449 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:48.405408 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:51.999665 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:51.999634 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:53.563011 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.562977 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t"] Jul 09 19:32:53.566670 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.566651 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.571173 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.571146 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a7212d\"/\"kube-root-ca.crt\"" Jul 09 19:32:53.571173 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.571163 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a7212d\"/\"default-dockercfg-5x4nb\"" Jul 09 19:32:53.571364 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.571146 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a7212d\"/\"openshift-service-ca.crt\"" Jul 09 19:32:53.571364 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.571148 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a7212d\"/\"test-model-a7212d-kserve-self-signed-certs\"" Jul 09 19:32:53.576646 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.576627 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t"] Jul 09 19:32:53.586278 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.586258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:32:53.662790 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.662758 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc468\" (UniqueName: \"kubernetes.io/projected/ba097035-8372-4831-9666-b3780ef2dbf7-kube-api-access-pc468\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.662790 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.662789 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-home\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.662969 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.662825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-model-cache\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.662969 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.662886 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-dshm\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.663055 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.662964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba097035-8372-4831-9666-b3780ef2dbf7-tls-certs\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.663055 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.662988 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-tmp-dir\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764024 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.763993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc468\" (UniqueName: \"kubernetes.io/projected/ba097035-8372-4831-9666-b3780ef2dbf7-kube-api-access-pc468\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764024 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-home\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-model-cache\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-dshm\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764116 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba097035-8372-4831-9666-b3780ef2dbf7-tls-certs\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-tmp-dir\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764525 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764485 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-model-cache\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764596 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764485 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-home\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.764596 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.764552 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-tmp-dir\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.766663 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.766644 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-dshm\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.766840 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.766823 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba097035-8372-4831-9666-b3780ef2dbf7-tls-certs\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.775110 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.775076 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc468\" (UniqueName: \"kubernetes.io/projected/ba097035-8372-4831-9666-b3780ef2dbf7-kube-api-access-pc468\") pod \"test-model-a7212d-kserve-69476bd5f5-wkj7t\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:53.877465 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:53.877360 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:32:54.018934 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:54.018911 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t"] Jul 09 19:32:54.021230 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:32:54.021199 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba097035_8372_4831_9666_b3780ef2dbf7.slice/crio-abb8143346ab67f4364a6f2cdd04a937f4a6a5fcea645d115ba8d89f0a02f95e WatchSource:0}: Error finding container abb8143346ab67f4364a6f2cdd04a937f4a6a5fcea645d115ba8d89f0a02f95e: Status 404 returned error can't find the container with id abb8143346ab67f4364a6f2cdd04a937f4a6a5fcea645d115ba8d89f0a02f95e Jul 09 19:32:54.871835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:54.871803 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" event={"ID":"ba097035-8372-4831-9666-b3780ef2dbf7","Type":"ContainerStarted","Data":"73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979"} Jul 09 19:32:54.871835 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:54.871846 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" event={"ID":"ba097035-8372-4831-9666-b3780ef2dbf7","Type":"ContainerStarted","Data":"abb8143346ab67f4364a6f2cdd04a937f4a6a5fcea645d115ba8d89f0a02f95e"} Jul 09 19:32:54.894086 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:32:54.894029 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" podStartSLOduration=1.894012995 podStartE2EDuration="1.894012995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:32:54.892862182 +0000 UTC m=+2075.344314639" watchObservedRunningTime="2026-07-09 19:32:54.894012995 +0000 UTC m=+2075.345465517" Jul 09 19:33:01.463764 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:01.463724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:01.554925 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:01.554894 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:03.877639 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:03.877606 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:33:03.878024 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:03.877898 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:33:03.890397 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:03.890375 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:33:03.926887 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:03.926863 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:33:09.796316 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:09.796273 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:09.895854 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:09.895802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:20.184483 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:20.184456 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:33:20.190375 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:20.190354 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:33:21.965009 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:21.964979 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:45.272824 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:45.272790 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t"] Jul 09 19:33:45.294556 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:45.294501 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:46.173627 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:46.173591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:46.328035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:46.327999 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:46.720275 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:46.720237 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:46.820166 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:46.820134 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:46.919856 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:46.919818 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:46.993431 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:46.993401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:33:51.798103 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:33:51.798066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:00.294203 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.294155 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" podUID="ba097035-8372-4831-9666-b3780ef2dbf7" containerName="main" containerID="cri-o://73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979" gracePeriod=45 Jul 09 19:34:00.536936 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.536912 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:34:00.634484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634393 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-dshm\") pod \"ba097035-8372-4831-9666-b3780ef2dbf7\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " Jul 09 19:34:00.634484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634436 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-model-cache\") pod \"ba097035-8372-4831-9666-b3780ef2dbf7\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " Jul 09 19:34:00.634484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634482 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-home\") pod \"ba097035-8372-4831-9666-b3780ef2dbf7\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " Jul 09 19:34:00.634786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634549 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc468\" (UniqueName: \"kubernetes.io/projected/ba097035-8372-4831-9666-b3780ef2dbf7-kube-api-access-pc468\") pod \"ba097035-8372-4831-9666-b3780ef2dbf7\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " Jul 09 19:34:00.634786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634578 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba097035-8372-4831-9666-b3780ef2dbf7-tls-certs\") pod \"ba097035-8372-4831-9666-b3780ef2dbf7\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " Jul 09 19:34:00.634786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634614 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-tmp-dir\") pod \"ba097035-8372-4831-9666-b3780ef2dbf7\" (UID: \"ba097035-8372-4831-9666-b3780ef2dbf7\") " Jul 09 19:34:00.634786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634751 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba097035-8372-4831-9666-b3780ef2dbf7" (UID: "ba097035-8372-4831-9666-b3780ef2dbf7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:34:00.634786 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634758 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-home" (OuterVolumeSpecName: "home") pod "ba097035-8372-4831-9666-b3780ef2dbf7" (UID: "ba097035-8372-4831-9666-b3780ef2dbf7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:34:00.635036 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.634943 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba097035-8372-4831-9666-b3780ef2dbf7" (UID: "ba097035-8372-4831-9666-b3780ef2dbf7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:34:00.635036 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.635013 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-tmp-dir\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:34:00.635036 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.635034 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-model-cache\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:34:00.635351 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.635046 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-home\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:34:00.637246 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.637220 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-dshm" (OuterVolumeSpecName: "dshm") pod "ba097035-8372-4831-9666-b3780ef2dbf7" (UID: "ba097035-8372-4831-9666-b3780ef2dbf7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:34:00.637376 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.637261 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba097035-8372-4831-9666-b3780ef2dbf7-kube-api-access-pc468" (OuterVolumeSpecName: "kube-api-access-pc468") pod "ba097035-8372-4831-9666-b3780ef2dbf7" (UID: "ba097035-8372-4831-9666-b3780ef2dbf7"). InnerVolumeSpecName "kube-api-access-pc468". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:34:00.637376 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.637300 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba097035-8372-4831-9666-b3780ef2dbf7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba097035-8372-4831-9666-b3780ef2dbf7" (UID: "ba097035-8372-4831-9666-b3780ef2dbf7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:34:00.736571 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.736542 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc468\" (UniqueName: \"kubernetes.io/projected/ba097035-8372-4831-9666-b3780ef2dbf7-kube-api-access-pc468\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:34:00.736571 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.736569 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba097035-8372-4831-9666-b3780ef2dbf7-tls-certs\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:34:00.736571 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:00.736579 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba097035-8372-4831-9666-b3780ef2dbf7-dshm\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:34:01.202112 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.202082 2561 generic.go:356] "Generic (PLEG): container finished" podID="ba097035-8372-4831-9666-b3780ef2dbf7" containerID="73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979" exitCode=0 Jul 09 19:34:01.202295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.202126 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" Jul 09 19:34:01.202295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.202169 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" event={"ID":"ba097035-8372-4831-9666-b3780ef2dbf7","Type":"ContainerDied","Data":"73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979"} Jul 09 19:34:01.202295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.202214 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t" event={"ID":"ba097035-8372-4831-9666-b3780ef2dbf7","Type":"ContainerDied","Data":"abb8143346ab67f4364a6f2cdd04a937f4a6a5fcea645d115ba8d89f0a02f95e"} Jul 09 19:34:01.202295 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.202236 2561 scope.go:117] "RemoveContainer" containerID="73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979" Jul 09 19:34:01.211401 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.211384 2561 scope.go:117] "RemoveContainer" containerID="73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979" Jul 09 19:34:01.211670 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:34:01.211653 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979\": container with ID starting with 73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979 not found: ID does not exist" containerID="73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979" Jul 09 19:34:01.211736 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.211679 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979"} err="failed to get container status \"73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979\": rpc error: code = NotFound desc = could not find container \"73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979\": container with ID starting with 73a4ec74ab64c5cbcae42d11bf79a60166b2ae82d47fd2059a5e641554340979 not found: ID does not exist" Jul 09 19:34:01.222963 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.222942 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t"] Jul 09 19:34:01.226491 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:01.226468 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a7212d/test-model-a7212d-kserve-69476bd5f5-wkj7t"] Jul 09 19:34:02.101214 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:02.101181 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba097035-8372-4831-9666-b3780ef2dbf7" path="/var/lib/kubelet/pods/ba097035-8372-4831-9666-b3780ef2dbf7/volumes" Jul 09 19:34:28.683198 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:28.683159 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:29.936291 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:29.936260 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:30.038537 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:30.038471 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:30.155581 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:30.155546 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:35.719412 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:35.719372 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:35.817737 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:35.817704 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:35.989198 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:35.989161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:36.216445 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:36.216413 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:36.599250 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:36.599209 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:36.865932 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:36.865848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:36.964322 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:36.964291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:40.172033 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:40.171997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:40.263749 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:40.263711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:40.343493 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:40.343455 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:40.639968 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:40.639921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:40.738228 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:40.738192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:40.837246 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:40.837213 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:44.670472 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:44.670436 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:45.121386 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:45.121355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:45.422986 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:45.422876 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:45.616665 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:45.616629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:48.215266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:48.215229 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:48.306311 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:48.306278 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:48.380208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:48.380175 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:48.864062 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:48.864030 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:48.963458 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:48.963422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:34:52.738144 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:34:52.738108 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:00.823978 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:00.823933 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:01.268951 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:01.268913 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:01.369054 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:01.369015 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:01.516601 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:01.516566 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:07.385153 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:07.385114 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:07.472493 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:07.472462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:07.675338 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:07.675259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:07.896561 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:07.896501 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:08.052727 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:08.052691 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:08.479760 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:08.479660 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:08.697190 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:08.697152 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:08.796806 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:08.796774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:08.925436 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:08.925401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:08.986739 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:08.986703 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:12.832657 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:12.832626 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:12.934404 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:12.934369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:14.348570 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:14.348535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:14.450035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:14.450006 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:14.622176 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:14.622097 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:14.832805 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:14.832771 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:15.179042 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:15.179010 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:15.499391 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:15.499357 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:15.807131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:15.807047 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:18.431063 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:18.431010 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:18.523638 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:18.523606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:18.606442 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:18.606411 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:18.862794 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:18.862750 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:19.002703 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:19.002665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:19.103456 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:19.103425 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:22.912950 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:22.912916 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:23.245770 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:23.245739 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:23.564565 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:23.564463 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:23.693172 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:23.693127 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:26.517853 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:26.517815 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:26.618944 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:26.618913 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:26.716869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:26.716834 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:26.809075 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:26.808980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:27.061498 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:27.061412 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:27.161756 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:27.161722 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:31.031132 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:31.031096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:40.310644 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:40.310603 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:40.408779 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:40.408748 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:48.716457 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:48.716364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:35:48.817162 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:35:48.817124 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:36:01.769446 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.769409 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d"] Jul 09 19:36:01.769941 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.769841 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba097035-8372-4831-9666-b3780ef2dbf7" containerName="main" Jul 09 19:36:01.769941 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.769853 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba097035-8372-4831-9666-b3780ef2dbf7" containerName="main" Jul 09 19:36:01.769941 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.769924 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba097035-8372-4831-9666-b3780ef2dbf7" containerName="main" Jul 09 19:36:01.773079 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.773060 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.777158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.777136 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"kube-root-ca.crt\"" Jul 09 19:36:01.777158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.777146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"default-dockercfg-l2hg6\"" Jul 09 19:36:01.777158 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.777141 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"auth-test-model-77ffac-kserve-self-signed-certs\"" Jul 09 19:36:01.777349 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.777229 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"openshift-service-ca.crt\"" Jul 09 19:36:01.782751 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.782731 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d"] Jul 09 19:36:01.794927 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.794898 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:36:01.836196 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.836164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tls-certs\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.836340 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.836251 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tmp-dir\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.836340 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.836285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-dshm\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.836340 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.836310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-model-cache\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.836447 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.836373 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24926\" (UniqueName: \"kubernetes.io/projected/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-kube-api-access-24926\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.836447 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.836430 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-home\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.937885 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.937852 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tmp-dir\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.937896 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-dshm\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.937916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-model-cache\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.937938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24926\" (UniqueName: \"kubernetes.io/projected/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-kube-api-access-24926\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.937985 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-home\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938048 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.938008 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tls-certs\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938331 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.938305 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tmp-dir\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938384 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.938364 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-model-cache\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.938436 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.938421 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-home\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.940317 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.940297 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-dshm\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.940467 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.940449 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tls-certs\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:01.946674 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:01.946650 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24926\" (UniqueName: \"kubernetes.io/projected/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-kube-api-access-24926\") pod \"auth-test-model-77ffac-kserve-69c9769b7f-fwb9d\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:02.083737 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:02.083631 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:02.223813 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:02.223779 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d"] Jul 09 19:36:02.225598 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:36:02.225569 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb034e9a6_cdab_457f_bcc3_60fec5c0ec16.slice/crio-26c56775bd12588b5ba42edd2201f24192db00f903f96b7654e68779ccacd961 WatchSource:0}: Error finding container 26c56775bd12588b5ba42edd2201f24192db00f903f96b7654e68779ccacd961: Status 404 returned error can't find the container with id 26c56775bd12588b5ba42edd2201f24192db00f903f96b7654e68779ccacd961 Jul 09 19:36:02.834359 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:02.834315 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" event={"ID":"b034e9a6-cdab-457f-bcc3-60fec5c0ec16","Type":"ContainerStarted","Data":"9be700c37c256dcb074d1b36c052626d19e1133e2753dac0ef07eced1c92c0fb"} Jul 09 19:36:02.834359 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:02.834367 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" event={"ID":"b034e9a6-cdab-457f-bcc3-60fec5c0ec16","Type":"ContainerStarted","Data":"26c56775bd12588b5ba42edd2201f24192db00f903f96b7654e68779ccacd961"} Jul 09 19:36:02.852207 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:02.852137 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" podStartSLOduration=1.852121339 podStartE2EDuration="1.852121339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:36:02.850242557 +0000 UTC m=+2263.301695028" watchObservedRunningTime="2026-07-09 19:36:02.852121339 +0000 UTC m=+2263.303573799" Jul 09 19:36:07.957260 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:07.957223 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:36:12.084699 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:12.084663 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:12.084699 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:12.084711 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:12.100608 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:12.100576 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:12.895327 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:12.895298 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:36:22.696382 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:22.696349 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:36:38.773297 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:38.773259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:36:38.874293 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:36:38.874260 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:37:19.633813 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:37:19.633729 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:38:20.218672 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:20.218643 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:38:20.231448 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:20.231414 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:38:22.456545 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:22.456494 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d"] Jul 09 19:38:22.474999 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:22.474972 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:38:23.261621 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:23.261590 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:38:30.082684 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:30.082645 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:38:37.478062 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.478011 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" podUID="b034e9a6-cdab-457f-bcc3-60fec5c0ec16" containerName="main" containerID="cri-o://9be700c37c256dcb074d1b36c052626d19e1133e2753dac0ef07eced1c92c0fb" gracePeriod=45 Jul 09 19:38:37.612212 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.612180 2561 generic.go:356] "Generic (PLEG): container finished" podID="b034e9a6-cdab-457f-bcc3-60fec5c0ec16" containerID="9be700c37c256dcb074d1b36c052626d19e1133e2753dac0ef07eced1c92c0fb" exitCode=0 Jul 09 19:38:37.612371 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.612262 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" event={"ID":"b034e9a6-cdab-457f-bcc3-60fec5c0ec16","Type":"ContainerDied","Data":"9be700c37c256dcb074d1b36c052626d19e1133e2753dac0ef07eced1c92c0fb"} Jul 09 19:38:37.730924 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.730906 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:38:37.860165 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860127 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-home\") pod \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " Jul 09 19:38:37.860363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860195 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-dshm\") pod \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " Jul 09 19:38:37.860363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860259 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tmp-dir\") pod \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " Jul 09 19:38:37.860363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860317 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-model-cache\") pod \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " Jul 09 19:38:37.860363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860344 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tls-certs\") pod \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " Jul 09 19:38:37.860596 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860394 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24926\" (UniqueName: \"kubernetes.io/projected/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-kube-api-access-24926\") pod \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\" (UID: \"b034e9a6-cdab-457f-bcc3-60fec5c0ec16\") " Jul 09 19:38:37.860596 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860407 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-home" (OuterVolumeSpecName: "home") pod "b034e9a6-cdab-457f-bcc3-60fec5c0ec16" (UID: "b034e9a6-cdab-457f-bcc3-60fec5c0ec16"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:38:37.860596 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860537 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-model-cache" (OuterVolumeSpecName: "model-cache") pod "b034e9a6-cdab-457f-bcc3-60fec5c0ec16" (UID: "b034e9a6-cdab-457f-bcc3-60fec5c0ec16"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:38:37.860596 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860584 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b034e9a6-cdab-457f-bcc3-60fec5c0ec16" (UID: "b034e9a6-cdab-457f-bcc3-60fec5c0ec16"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:38:37.860825 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860806 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-home\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:38:37.860881 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860831 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tmp-dir\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:38:37.860881 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.860846 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-model-cache\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:38:37.862707 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.862679 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-dshm" (OuterVolumeSpecName: "dshm") pod "b034e9a6-cdab-457f-bcc3-60fec5c0ec16" (UID: "b034e9a6-cdab-457f-bcc3-60fec5c0ec16"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:38:37.862707 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.862688 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-kube-api-access-24926" (OuterVolumeSpecName: "kube-api-access-24926") pod "b034e9a6-cdab-457f-bcc3-60fec5c0ec16" (UID: "b034e9a6-cdab-457f-bcc3-60fec5c0ec16"). InnerVolumeSpecName "kube-api-access-24926". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:38:37.862898 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.862747 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b034e9a6-cdab-457f-bcc3-60fec5c0ec16" (UID: "b034e9a6-cdab-457f-bcc3-60fec5c0ec16"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:38:37.961997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.961960 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-tls-certs\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:38:37.961997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.961990 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24926\" (UniqueName: \"kubernetes.io/projected/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-kube-api-access-24926\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:38:37.961997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:37.962000 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b034e9a6-cdab-457f-bcc3-60fec5c0ec16-dshm\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:38:38.618986 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:38.618955 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" Jul 09 19:38:38.619386 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:38.618961 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d" event={"ID":"b034e9a6-cdab-457f-bcc3-60fec5c0ec16","Type":"ContainerDied","Data":"26c56775bd12588b5ba42edd2201f24192db00f903f96b7654e68779ccacd961"} Jul 09 19:38:38.619386 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:38.619086 2561 scope.go:117] "RemoveContainer" containerID="9be700c37c256dcb074d1b36c052626d19e1133e2753dac0ef07eced1c92c0fb" Jul 09 19:38:38.636070 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:38.636043 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d"] Jul 09 19:38:38.640065 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:38.640042 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/auth-test-model-77ffac-kserve-69c9769b7f-fwb9d"] Jul 09 19:38:40.100569 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:40.100529 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b034e9a6-cdab-457f-bcc3-60fec5c0ec16" path="/var/lib/kubelet/pods/b034e9a6-cdab-457f-bcc3-60fec5c0ec16/volumes" Jul 09 19:38:58.771658 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.771580 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx"] Jul 09 19:38:58.772136 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.772065 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b034e9a6-cdab-457f-bcc3-60fec5c0ec16" containerName="main" Jul 09 19:38:58.772136 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.772080 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b034e9a6-cdab-457f-bcc3-60fec5c0ec16" containerName="main" Jul 09 19:38:58.772208 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.772140 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b034e9a6-cdab-457f-bcc3-60fec5c0ec16" containerName="main" Jul 09 19:38:58.775203 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.775187 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.778879 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.778850 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"kube-root-ca.crt\"" Jul 09 19:38:58.778879 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.778870 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"openshift-service-ca.crt\"" Jul 09 19:38:58.779186 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.778854 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"sub-test-model-11fb58-kserve-self-signed-certs\"" Jul 09 19:38:58.779186 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.778877 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"default-dockercfg-rfv8s\"" Jul 09 19:38:58.784846 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.784824 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx"] Jul 09 19:38:58.793745 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.793717 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:38:58.865169 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.865128 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xjlm\" (UniqueName: \"kubernetes.io/projected/83e235f1-b683-4bac-93b1-a17a5066266b-kube-api-access-4xjlm\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.865378 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.865224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-tmp-dir\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.865378 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.865254 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-model-cache\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.865378 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.865289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-dshm\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.865378 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.865339 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e235f1-b683-4bac-93b1-a17a5066266b-tls-certs\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.865591 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.865398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-home\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966450 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-tmp-dir\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966450 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966451 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-model-cache\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966478 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-dshm\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966535 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e235f1-b683-4bac-93b1-a17a5066266b-tls-certs\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-home\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966726 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xjlm\" (UniqueName: \"kubernetes.io/projected/83e235f1-b683-4bac-93b1-a17a5066266b-kube-api-access-4xjlm\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.966949 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966892 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-tmp-dir\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.967003 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.966959 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-model-cache\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.967081 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.967059 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-home\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.969005 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.968978 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-dshm\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.969157 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.969139 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e235f1-b683-4bac-93b1-a17a5066266b-tls-certs\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:58.974866 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:58.974845 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xjlm\" (UniqueName: \"kubernetes.io/projected/83e235f1-b683-4bac-93b1-a17a5066266b-kube-api-access-4xjlm\") pod \"sub-test-model-11fb58-kserve-5476b44df7-csqpx\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:59.085367 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:59.085266 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:38:59.225607 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:59.225544 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx"] Jul 09 19:38:59.227965 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:38:59.227919 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83e235f1_b683_4bac_93b1_a17a5066266b.slice/crio-39e63ae9d28d27c053130033de51776dcbd7e20a235dbc580070ef195f0b2c85 WatchSource:0}: Error finding container 39e63ae9d28d27c053130033de51776dcbd7e20a235dbc580070ef195f0b2c85: Status 404 returned error can't find the container with id 39e63ae9d28d27c053130033de51776dcbd7e20a235dbc580070ef195f0b2c85 Jul 09 19:38:59.445241 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:59.445146 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:38:59.725408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:59.725308 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" event={"ID":"83e235f1-b683-4bac-93b1-a17a5066266b","Type":"ContainerStarted","Data":"c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8"} Jul 09 19:38:59.725408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:59.725353 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" event={"ID":"83e235f1-b683-4bac-93b1-a17a5066266b","Type":"ContainerStarted","Data":"39e63ae9d28d27c053130033de51776dcbd7e20a235dbc580070ef195f0b2c85"} Jul 09 19:38:59.745813 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:38:59.745762 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" podStartSLOduration=1.745746602 podStartE2EDuration="1.745746602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:38:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:38:59.744032022 +0000 UTC m=+2440.195484485" watchObservedRunningTime="2026-07-09 19:38:59.745746602 +0000 UTC m=+2440.197199062" Jul 09 19:39:09.086049 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:09.086002 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:39:09.086546 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:09.086285 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:39:09.099141 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:09.099114 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:39:09.786476 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:09.786447 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:39:44.464414 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:44.464377 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:39:45.389739 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:45.389693 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx"] Jul 09 19:39:45.639348 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:45.639314 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:39:59.582011 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.581977 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88"] Jul 09 19:39:59.585653 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.585635 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.590198 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.590172 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"kube-root-ca.crt\"" Jul 09 19:39:59.593419 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.593397 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"default-dockercfg-l2hg6\"" Jul 09 19:39:59.593543 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.593479 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"rate-test-model-77ffac-kserve-self-signed-certs\"" Jul 09 19:39:59.593543 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.593534 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77ffac\"/\"openshift-service-ca.crt\"" Jul 09 19:39:59.602902 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.602879 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88"] Jul 09 19:39:59.608181 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.608155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:39:59.761839 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.761812 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-model-cache\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.762005 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.761843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/4391e687-a4b6-4a60-bffe-1e6f6399bc31-kube-api-access-np2s5\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.762005 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.761864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tmp-dir\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.762005 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.761969 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-home\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.762005 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.761995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tls-certs\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.762143 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.762030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-dshm\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863123 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-model-cache\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863123 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863080 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/4391e687-a4b6-4a60-bffe-1e6f6399bc31-kube-api-access-np2s5\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863123 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tmp-dir\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-home\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863363 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tls-certs\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863419 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tmp-dir\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-dshm\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863615 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863481 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-model-cache\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.863615 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.863580 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-home\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.865623 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.865594 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-dshm\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.865869 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.865853 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tls-certs\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.871388 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.871369 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/4391e687-a4b6-4a60-bffe-1e6f6399bc31-kube-api-access-np2s5\") pod \"rate-test-model-77ffac-kserve-5bbb95db7f-prh88\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:39:59.895377 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:39:59.895350 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:40:00.064934 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.064901 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88"] Jul 09 19:40:00.068648 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:40:00.068619 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4391e687_a4b6_4a60_bffe_1e6f6399bc31.slice/crio-203155c1a948c1d957eacfe9ceddc22dd58faa53e8790f6e42190aeb9960cf69 WatchSource:0}: Error finding container 203155c1a948c1d957eacfe9ceddc22dd58faa53e8790f6e42190aeb9960cf69: Status 404 returned error can't find the container with id 203155c1a948c1d957eacfe9ceddc22dd58faa53e8790f6e42190aeb9960cf69 Jul 09 19:40:00.410560 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.410454 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" podUID="83e235f1-b683-4bac-93b1-a17a5066266b" containerName="main" containerID="cri-o://c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8" gracePeriod=45 Jul 09 19:40:00.647394 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.647367 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:40:00.773834 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.773794 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-home\") pod \"83e235f1-b683-4bac-93b1-a17a5066266b\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " Jul 09 19:40:00.773834 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.773842 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-dshm\") pod \"83e235f1-b683-4bac-93b1-a17a5066266b\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " Jul 09 19:40:00.774094 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.773895 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xjlm\" (UniqueName: \"kubernetes.io/projected/83e235f1-b683-4bac-93b1-a17a5066266b-kube-api-access-4xjlm\") pod \"83e235f1-b683-4bac-93b1-a17a5066266b\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " Jul 09 19:40:00.774094 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.773948 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-tmp-dir\") pod \"83e235f1-b683-4bac-93b1-a17a5066266b\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " Jul 09 19:40:00.774094 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774016 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-model-cache\") pod \"83e235f1-b683-4bac-93b1-a17a5066266b\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " Jul 09 19:40:00.774294 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774094 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e235f1-b683-4bac-93b1-a17a5066266b-tls-certs\") pod \"83e235f1-b683-4bac-93b1-a17a5066266b\" (UID: \"83e235f1-b683-4bac-93b1-a17a5066266b\") " Jul 09 19:40:00.774294 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774096 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-home" (OuterVolumeSpecName: "home") pod "83e235f1-b683-4bac-93b1-a17a5066266b" (UID: "83e235f1-b683-4bac-93b1-a17a5066266b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:40:00.774428 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774289 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83e235f1-b683-4bac-93b1-a17a5066266b" (UID: "83e235f1-b683-4bac-93b1-a17a5066266b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:40:00.774428 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774314 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-model-cache" (OuterVolumeSpecName: "model-cache") pod "83e235f1-b683-4bac-93b1-a17a5066266b" (UID: "83e235f1-b683-4bac-93b1-a17a5066266b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:40:00.774793 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774747 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-tmp-dir\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:40:00.774793 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774771 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-model-cache\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:40:00.774793 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.774781 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-home\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:40:00.776350 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.776330 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e235f1-b683-4bac-93b1-a17a5066266b-kube-api-access-4xjlm" (OuterVolumeSpecName: "kube-api-access-4xjlm") pod "83e235f1-b683-4bac-93b1-a17a5066266b" (UID: "83e235f1-b683-4bac-93b1-a17a5066266b"). InnerVolumeSpecName "kube-api-access-4xjlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:40:00.776732 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.776714 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83e235f1-b683-4bac-93b1-a17a5066266b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83e235f1-b683-4bac-93b1-a17a5066266b" (UID: "83e235f1-b683-4bac-93b1-a17a5066266b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:40:00.776732 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.776726 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-dshm" (OuterVolumeSpecName: "dshm") pod "83e235f1-b683-4bac-93b1-a17a5066266b" (UID: "83e235f1-b683-4bac-93b1-a17a5066266b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:40:00.875846 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.875812 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e235f1-b683-4bac-93b1-a17a5066266b-tls-certs\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:40:00.875970 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.875849 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e235f1-b683-4bac-93b1-a17a5066266b-dshm\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:40:00.875970 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:00.875862 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xjlm\" (UniqueName: \"kubernetes.io/projected/83e235f1-b683-4bac-93b1-a17a5066266b-kube-api-access-4xjlm\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:40:01.045484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.045398 2561 generic.go:356] "Generic (PLEG): container finished" podID="83e235f1-b683-4bac-93b1-a17a5066266b" containerID="c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8" exitCode=0 Jul 09 19:40:01.045484 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.045441 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" Jul 09 19:40:01.045713 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.045481 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" event={"ID":"83e235f1-b683-4bac-93b1-a17a5066266b","Type":"ContainerDied","Data":"c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8"} Jul 09 19:40:01.045713 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.045543 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx" event={"ID":"83e235f1-b683-4bac-93b1-a17a5066266b","Type":"ContainerDied","Data":"39e63ae9d28d27c053130033de51776dcbd7e20a235dbc580070ef195f0b2c85"} Jul 09 19:40:01.045713 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.045567 2561 scope.go:117] "RemoveContainer" containerID="c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8" Jul 09 19:40:01.047128 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.047099 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" event={"ID":"4391e687-a4b6-4a60-bffe-1e6f6399bc31","Type":"ContainerStarted","Data":"b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5"} Jul 09 19:40:01.047262 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.047134 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" event={"ID":"4391e687-a4b6-4a60-bffe-1e6f6399bc31","Type":"ContainerStarted","Data":"203155c1a948c1d957eacfe9ceddc22dd58faa53e8790f6e42190aeb9960cf69"} Jul 09 19:40:01.055755 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.055738 2561 scope.go:117] "RemoveContainer" containerID="c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8" Jul 09 19:40:01.056002 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:40:01.055984 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8\": container with ID starting with c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8 not found: ID does not exist" containerID="c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8" Jul 09 19:40:01.056055 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.056010 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8"} err="failed to get container status \"c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8\": rpc error: code = NotFound desc = could not find container \"c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8\": container with ID starting with c03728a8fd9a6dc4f2702db71481c075bf322605f02afa6e590cc827782a41e8 not found: ID does not exist" Jul 09 19:40:01.066911 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.066869 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" podStartSLOduration=2.066853514 podStartE2EDuration="2.066853514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:01.065999873 +0000 UTC m=+2501.517452345" watchObservedRunningTime="2026-07-09 19:40:01.066853514 +0000 UTC m=+2501.518306031" Jul 09 19:40:01.080419 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.080391 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx"] Jul 09 19:40:01.084594 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:01.084570 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/sub-test-model-11fb58-kserve-5476b44df7-csqpx"] Jul 09 19:40:02.100930 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:02.100895 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83e235f1-b683-4bac-93b1-a17a5066266b" path="/var/lib/kubelet/pods/83e235f1-b683-4bac-93b1-a17a5066266b/volumes" Jul 09 19:40:09.896311 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:09.896272 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:40:09.896847 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:09.896331 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:40:09.908951 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:09.908921 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:40:10.105114 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:10.105083 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:40:13.668067 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:13.667934 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:13.768083 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:13.768049 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:41.324620 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:41.324579 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:41.426206 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:41.426173 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:43.167189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.167161 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb"] Jul 09 19:40:43.167663 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.167594 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e235f1-b683-4bac-93b1-a17a5066266b" containerName="main" Jul 09 19:40:43.167663 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.167611 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e235f1-b683-4bac-93b1-a17a5066266b" containerName="main" Jul 09 19:40:43.167755 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.167710 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="83e235f1-b683-4bac-93b1-a17a5066266b" containerName="main" Jul 09 19:40:43.170766 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.170749 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.174338 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.174311 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"openshift-service-ca.crt\"" Jul 09 19:40:43.174459 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.174358 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"kube-root-ca.crt\"" Jul 09 19:40:43.174459 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.174314 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"default-dockercfg-rfv8s\"" Jul 09 19:40:43.174459 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.174314 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-11fb58\"/\"rate-test-model-11fb58-kserve-self-signed-certs\"" Jul 09 19:40:43.180982 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.180962 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb"] Jul 09 19:40:43.198390 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.198366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-model-cache\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.198526 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.198402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-home\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.198526 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.198423 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5dbz\" (UniqueName: \"kubernetes.io/projected/73444b66-b3e2-494b-8620-452df7484ad0-kube-api-access-b5dbz\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.198526 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.198449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73444b66-b3e2-494b-8620-452df7484ad0-tls-certs\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.198526 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.198498 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-tmp-dir\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.198692 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.198574 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-dshm\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.258664 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.258629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:43.299675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.299642 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-home\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.299867 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.299683 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5dbz\" (UniqueName: \"kubernetes.io/projected/73444b66-b3e2-494b-8620-452df7484ad0-kube-api-access-b5dbz\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.299867 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.299813 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73444b66-b3e2-494b-8620-452df7484ad0-tls-certs\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.300013 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.299882 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-tmp-dir\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.300013 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.299921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-dshm\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.300131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.300039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-model-cache\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.300189 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.300176 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-home\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.300235 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.300211 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-tmp-dir\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.300374 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.300349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-model-cache\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.302299 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.302269 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-dshm\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.302389 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.302344 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73444b66-b3e2-494b-8620-452df7484ad0-tls-certs\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.307720 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.307697 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5dbz\" (UniqueName: \"kubernetes.io/projected/73444b66-b3e2-494b-8620-452df7484ad0-kube-api-access-b5dbz\") pod \"rate-test-model-11fb58-kserve-7dc4fb895f-6knqb\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.480931 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.480899 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:43.618729 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:43.618696 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb"] Jul 09 19:40:43.621485 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:40:43.621458 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73444b66_b3e2_494b_8620_452df7484ad0.slice/crio-96924459d10398b35b29739b783dbadd0d95420769ddd3fd43cc66c7e07be119 WatchSource:0}: Error finding container 96924459d10398b35b29739b783dbadd0d95420769ddd3fd43cc66c7e07be119: Status 404 returned error can't find the container with id 96924459d10398b35b29739b783dbadd0d95420769ddd3fd43cc66c7e07be119 Jul 09 19:40:44.268950 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:44.268912 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" event={"ID":"73444b66-b3e2-494b-8620-452df7484ad0","Type":"ContainerStarted","Data":"c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06"} Jul 09 19:40:44.269355 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:44.268960 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" event={"ID":"73444b66-b3e2-494b-8620-452df7484ad0","Type":"ContainerStarted","Data":"96924459d10398b35b29739b783dbadd0d95420769ddd3fd43cc66c7e07be119"} Jul 09 19:40:44.288240 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:44.288191 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" podStartSLOduration=1.288175018 podStartE2EDuration="1.288175018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:44.285665683 +0000 UTC m=+2544.737118146" watchObservedRunningTime="2026-07-09 19:40:44.288175018 +0000 UTC m=+2544.739627478" Jul 09 19:40:46.187409 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:46.187369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:49.721852 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:49.721813 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:49.821849 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:49.821771 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:40:53.481270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:53.481224 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:53.481270 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:53.481287 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:53.493684 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:53.493661 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:40:54.330908 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:40:54.330881 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:41:27.490804 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:27.490768 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88"] Jul 09 19:41:27.512203 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:27.512173 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:28.454434 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:28.454397 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb"] Jul 09 19:41:28.473644 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:28.473613 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:42.512405 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.512317 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" podUID="4391e687-a4b6-4a60-bffe-1e6f6399bc31" containerName="main" containerID="cri-o://b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5" gracePeriod=45 Jul 09 19:41:42.773341 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.773279 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:41:42.911045 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.910992 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-dshm\") pod \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " Jul 09 19:41:42.911266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911095 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/4391e687-a4b6-4a60-bffe-1e6f6399bc31-kube-api-access-np2s5\") pod \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " Jul 09 19:41:42.911266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911152 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-home\") pod \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " Jul 09 19:41:42.911266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911191 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-model-cache\") pod \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " Jul 09 19:41:42.911266 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911252 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tls-certs\") pod \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " Jul 09 19:41:42.911474 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911324 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tmp-dir\") pod \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\" (UID: \"4391e687-a4b6-4a60-bffe-1e6f6399bc31\") " Jul 09 19:41:42.911557 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911466 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-model-cache" (OuterVolumeSpecName: "model-cache") pod "4391e687-a4b6-4a60-bffe-1e6f6399bc31" (UID: "4391e687-a4b6-4a60-bffe-1e6f6399bc31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:42.911557 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911479 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-home" (OuterVolumeSpecName: "home") pod "4391e687-a4b6-4a60-bffe-1e6f6399bc31" (UID: "4391e687-a4b6-4a60-bffe-1e6f6399bc31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:42.911682 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911646 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4391e687-a4b6-4a60-bffe-1e6f6399bc31" (UID: "4391e687-a4b6-4a60-bffe-1e6f6399bc31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:42.911988 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911973 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tmp-dir\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:42.912057 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.911994 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-home\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:42.912057 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.912007 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-model-cache\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:42.913424 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.913398 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-dshm" (OuterVolumeSpecName: "dshm") pod "4391e687-a4b6-4a60-bffe-1e6f6399bc31" (UID: "4391e687-a4b6-4a60-bffe-1e6f6399bc31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:42.913545 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.913422 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4391e687-a4b6-4a60-bffe-1e6f6399bc31-kube-api-access-np2s5" (OuterVolumeSpecName: "kube-api-access-np2s5") pod "4391e687-a4b6-4a60-bffe-1e6f6399bc31" (UID: "4391e687-a4b6-4a60-bffe-1e6f6399bc31"). InnerVolumeSpecName "kube-api-access-np2s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:41:42.913545 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:42.913457 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4391e687-a4b6-4a60-bffe-1e6f6399bc31" (UID: "4391e687-a4b6-4a60-bffe-1e6f6399bc31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:41:43.013475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.013429 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/4391e687-a4b6-4a60-bffe-1e6f6399bc31-kube-api-access-np2s5\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:43.013475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.013466 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4391e687-a4b6-4a60-bffe-1e6f6399bc31-tls-certs\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:43.013475 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.013479 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4391e687-a4b6-4a60-bffe-1e6f6399bc31-dshm\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:43.475330 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.475260 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" podUID="73444b66-b3e2-494b-8620-452df7484ad0" containerName="main" containerID="cri-o://c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06" gracePeriod=45 Jul 09 19:41:43.570711 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.570675 2561 generic.go:356] "Generic (PLEG): container finished" podID="4391e687-a4b6-4a60-bffe-1e6f6399bc31" containerID="b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5" exitCode=0 Jul 09 19:41:43.571066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.570722 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" Jul 09 19:41:43.571066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.570754 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" event={"ID":"4391e687-a4b6-4a60-bffe-1e6f6399bc31","Type":"ContainerDied","Data":"b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5"} Jul 09 19:41:43.571066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.570800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88" event={"ID":"4391e687-a4b6-4a60-bffe-1e6f6399bc31","Type":"ContainerDied","Data":"203155c1a948c1d957eacfe9ceddc22dd58faa53e8790f6e42190aeb9960cf69"} Jul 09 19:41:43.571066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.570817 2561 scope.go:117] "RemoveContainer" containerID="b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5" Jul 09 19:41:43.590165 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.590095 2561 scope.go:117] "RemoveContainer" containerID="b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5" Jul 09 19:41:43.590457 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:41:43.590434 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5\": container with ID starting with b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5 not found: ID does not exist" containerID="b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5" Jul 09 19:41:43.590545 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.590463 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5"} err="failed to get container status \"b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5\": rpc error: code = NotFound desc = could not find container \"b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5\": container with ID starting with b85d7fc21e548863927735b7baa9f72c49368fb418646a20fe5af61a1be973d5 not found: ID does not exist" Jul 09 19:41:43.592056 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.592033 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88"] Jul 09 19:41:43.594029 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.594006 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-77ffac/rate-test-model-77ffac-kserve-5bbb95db7f-prh88"] Jul 09 19:41:43.732612 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.732588 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:41:43.923650 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.923615 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-home\") pod \"73444b66-b3e2-494b-8620-452df7484ad0\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " Jul 09 19:41:43.923650 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.923653 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-tmp-dir\") pod \"73444b66-b3e2-494b-8620-452df7484ad0\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " Jul 09 19:41:43.923872 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.923703 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-model-cache\") pod \"73444b66-b3e2-494b-8620-452df7484ad0\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " Jul 09 19:41:43.923872 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.923775 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73444b66-b3e2-494b-8620-452df7484ad0-tls-certs\") pod \"73444b66-b3e2-494b-8620-452df7484ad0\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " Jul 09 19:41:43.923872 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.923813 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5dbz\" (UniqueName: \"kubernetes.io/projected/73444b66-b3e2-494b-8620-452df7484ad0-kube-api-access-b5dbz\") pod \"73444b66-b3e2-494b-8620-452df7484ad0\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " Jul 09 19:41:43.923872 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.923857 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-dshm\") pod \"73444b66-b3e2-494b-8620-452df7484ad0\" (UID: \"73444b66-b3e2-494b-8620-452df7484ad0\") " Jul 09 19:41:43.924077 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.924000 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-home" (OuterVolumeSpecName: "home") pod "73444b66-b3e2-494b-8620-452df7484ad0" (UID: "73444b66-b3e2-494b-8620-452df7484ad0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:43.924077 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.924015 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-model-cache" (OuterVolumeSpecName: "model-cache") pod "73444b66-b3e2-494b-8620-452df7484ad0" (UID: "73444b66-b3e2-494b-8620-452df7484ad0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:43.924077 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.924026 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73444b66-b3e2-494b-8620-452df7484ad0" (UID: "73444b66-b3e2-494b-8620-452df7484ad0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:43.924207 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.924198 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-home\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:43.924243 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.924210 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-tmp-dir\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:43.924243 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.924224 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-model-cache\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:43.926095 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.926059 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73444b66-b3e2-494b-8620-452df7484ad0-kube-api-access-b5dbz" (OuterVolumeSpecName: "kube-api-access-b5dbz") pod "73444b66-b3e2-494b-8620-452df7484ad0" (UID: "73444b66-b3e2-494b-8620-452df7484ad0"). InnerVolumeSpecName "kube-api-access-b5dbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:41:43.926095 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.926074 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73444b66-b3e2-494b-8620-452df7484ad0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73444b66-b3e2-494b-8620-452df7484ad0" (UID: "73444b66-b3e2-494b-8620-452df7484ad0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 19:41:43.926095 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:43.926094 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-dshm" (OuterVolumeSpecName: "dshm") pod "73444b66-b3e2-494b-8620-452df7484ad0" (UID: "73444b66-b3e2-494b-8620-452df7484ad0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:41:44.025306 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.025207 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73444b66-b3e2-494b-8620-452df7484ad0-tls-certs\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:44.025306 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.025250 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5dbz\" (UniqueName: \"kubernetes.io/projected/73444b66-b3e2-494b-8620-452df7484ad0-kube-api-access-b5dbz\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:44.025306 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.025264 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73444b66-b3e2-494b-8620-452df7484ad0-dshm\") on node \"ip-10-0-133-17.ec2.internal\" DevicePath \"\"" Jul 09 19:41:44.101413 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.101376 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4391e687-a4b6-4a60-bffe-1e6f6399bc31" path="/var/lib/kubelet/pods/4391e687-a4b6-4a60-bffe-1e6f6399bc31/volumes" Jul 09 19:41:44.579141 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.579106 2561 generic.go:356] "Generic (PLEG): container finished" podID="73444b66-b3e2-494b-8620-452df7484ad0" containerID="c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06" exitCode=0 Jul 09 19:41:44.579141 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.579144 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" event={"ID":"73444b66-b3e2-494b-8620-452df7484ad0","Type":"ContainerDied","Data":"c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06"} Jul 09 19:41:44.579674 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.579151 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" Jul 09 19:41:44.579674 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.579174 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb" event={"ID":"73444b66-b3e2-494b-8620-452df7484ad0","Type":"ContainerDied","Data":"96924459d10398b35b29739b783dbadd0d95420769ddd3fd43cc66c7e07be119"} Jul 09 19:41:44.579674 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.579191 2561 scope.go:117] "RemoveContainer" containerID="c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06" Jul 09 19:41:44.588302 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.588285 2561 scope.go:117] "RemoveContainer" containerID="c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06" Jul 09 19:41:44.588572 ip-10-0-133-17 kubenswrapper[2561]: E0709 19:41:44.588552 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06\": container with ID starting with c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06 not found: ID does not exist" containerID="c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06" Jul 09 19:41:44.588629 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.588581 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06"} err="failed to get container status \"c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06\": rpc error: code = NotFound desc = could not find container \"c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06\": container with ID starting with c73acb9598f65735027a54d73051a49173bd030b52676e9f830845d6beb4ec06 not found: ID does not exist" Jul 09 19:41:44.597094 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.597070 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb"] Jul 09 19:41:44.600675 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:44.600653 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-11fb58/rate-test-model-11fb58-kserve-7dc4fb895f-6knqb"] Jul 09 19:41:46.102135 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:46.102098 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73444b66-b3e2-494b-8620-452df7484ad0" path="/var/lib/kubelet/pods/73444b66-b3e2-494b-8620-452df7484ad0/volumes" Jul 09 19:41:48.172814 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:48.172785 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:48.566402 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:48.566365 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:48.683855 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:48.683816 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:48.793356 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:48.793327 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:49.002987 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:49.002959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:54.812656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:54.812609 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:54.879470 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:54.879437 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:55.128763 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:55.128677 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:55.348213 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:55.348176 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:55.458762 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:55.458676 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:55.856312 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:55.856279 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:55.955868 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:55.955836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:56.056251 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:56.056220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:41:56.133069 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:41:56.132982 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:01.921032 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:01.920980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:02.021678 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:02.021637 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:02.171588 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:02.171489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:02.269377 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:02.269346 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:02.328323 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:02.328291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:02.399166 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:02.399136 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:07.715434 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:07.715399 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5856874589-nv5m5_1680d844-21aa-4977-9e0c-86e88dc05612/manager/0.log" Jul 09 19:42:08.061142 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:08.061116 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-799d55cdcd-7l5vd_1030534d-6078-49af-9143-88d95a426541/copy-manifests/0.log" Jul 09 19:42:08.068025 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:08.067991 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-799d55cdcd-7l5vd_1030534d-6078-49af-9143-88d95a426541/manager/0.log" Jul 09 19:42:08.631600 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:08.631570 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8486c54598-2xkbc_52905b97-b7e2-40b6-901f-f8c45c2c8414/maas-api/0.log" Jul 09 19:42:08.743167 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:08.743138 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5lsg6_3cba549a-60bc-4a35-a5f8-09b405c05596/postgres/0.log" Jul 09 19:42:10.200798 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:10.200773 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p8pb7_b63ef606-a837-47df-a988-90925356b044/manager/0.log" Jul 09 19:42:10.313066 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:10.313041 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-65hvw_97420cc8-4422-4848-b14c-ed3e395fcb1b/kuadrant-console-plugin/0.log" Jul 09 19:42:11.374212 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:11.374180 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-db49944d8-gdk2c_e1cdebdb-59c4-4e69-879c-4198a565cc8a/kube-auth-proxy/0.log" Jul 09 19:42:11.728113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:11.728034 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-cjnxj_8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1/payload-pre-processing/0.log" Jul 09 19:42:11.947958 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:11.947931 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c45df5c76-pqvtd_1b789612-d416-4ae7-880e-71c9bc9939e3/router/0.log" Jul 09 19:42:12.277013 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:12.276979 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-z7xkj_5da3c67c-b571-4fd4-a48b-364d8d9e7166/main/0.log" Jul 09 19:42:12.483113 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:12.483086 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-prvk8_b9656346-57ae-4671-9950-f54388b0f2aa/main/0.log" Jul 09 19:42:12.698612 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:12.698536 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hzfkr_3e668f44-e6ea-4b3d-b6ac-70cc0177b489/main/0.log" Jul 09 19:42:12.807951 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:12.807915 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-twhl9_08558249-01ac-45f9-87b3-83b82fef61f9/main/0.log" Jul 09 19:42:16.616389 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:16.616357 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:16.717119 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:16.717083 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:19.216131 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:19.216100 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jfmg2_52c8d752-0a4d-4e02-85f9-435185c3da3d/global-pull-secret-syncer/0.log" Jul 09 19:42:19.346601 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:19.346573 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" Jul 09 19:42:19.428942 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:19.428899 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-17.ec2.internal_2167be909be366b8f502c0ed2d1d4ffd/haproxy/0.log" Jul 09 19:42:24.001324 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:24.001291 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p8pb7_b63ef606-a837-47df-a988-90925356b044/manager/0.log" Jul 09 19:42:24.028112 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:24.028085 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-65hvw_97420cc8-4422-4848-b14c-ed3e395fcb1b/kuadrant-console-plugin/0.log" Jul 09 19:42:25.982656 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:25.982627 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/alertmanager/0.log" Jul 09 19:42:26.004371 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.004349 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/config-reloader/0.log" Jul 09 19:42:26.027056 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.027022 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/kube-rbac-proxy-web/0.log" Jul 09 19:42:26.058035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.058005 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/kube-rbac-proxy/0.log" Jul 09 19:42:26.079637 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.079612 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/kube-rbac-proxy-metric/0.log" Jul 09 19:42:26.100845 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.100824 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/prom-label-proxy/0.log" Jul 09 19:42:26.123972 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.123949 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_795af1df-4fb9-4712-9c0d-8e50c35614c3/init-config-reloader/0.log" Jul 09 19:42:26.159441 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.159412 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-cn5rz_306d86b2-299c-46f3-b01d-caf74eb20c23/cluster-monitoring-operator/0.log" Jul 09 19:42:26.336004 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.335979 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6lcd7_ce2d9570-2634-4174-a950-c8488f5004c2/node-exporter/0.log" Jul 09 19:42:26.356972 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.356933 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6lcd7_ce2d9570-2634-4174-a950-c8488f5004c2/kube-rbac-proxy/0.log" Jul 09 19:42:26.378370 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.378349 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6lcd7_ce2d9570-2634-4174-a950-c8488f5004c2/init-textfile/0.log" Jul 09 19:42:26.720130 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.720102 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/prometheus/0.log" Jul 09 19:42:26.742191 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.742168 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/config-reloader/0.log" Jul 09 19:42:26.765520 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.765488 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/thanos-sidecar/0.log" Jul 09 19:42:26.787799 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.787771 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/kube-rbac-proxy-web/0.log" Jul 09 19:42:26.809229 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.809203 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/kube-rbac-proxy/0.log" Jul 09 19:42:26.832005 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.831983 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/kube-rbac-proxy-thanos/0.log" Jul 09 19:42:26.853662 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.853596 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3ddf050e-f986-4391-a075-ca379be9fa9a/init-config-reloader/0.log" Jul 09 19:42:26.888124 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.888097 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-wfphb_d8a09f1e-f229-4f72-b717-838d5b300f1c/prometheus-operator/0.log" Jul 09 19:42:26.913387 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.913361 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-wfphb_d8a09f1e-f229-4f72-b717-838d5b300f1c/kube-rbac-proxy/0.log" Jul 09 19:42:26.999973 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:26.999947 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f8bd55d44-95sdk_439e538c-a3ca-46ef-8ec9-4953a524ffaf/telemeter-client/0.log" Jul 09 19:42:27.027967 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.027920 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f8bd55d44-95sdk_439e538c-a3ca-46ef-8ec9-4953a524ffaf/reload/0.log" Jul 09 19:42:27.055186 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.055166 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f8bd55d44-95sdk_439e538c-a3ca-46ef-8ec9-4953a524ffaf/kube-rbac-proxy/0.log" Jul 09 19:42:27.088446 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.088416 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57ffb4c87-sfxhb_95856664-6f2f-4aec-bfad-23bb25c553df/thanos-query/0.log" Jul 09 19:42:27.114639 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.114573 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57ffb4c87-sfxhb_95856664-6f2f-4aec-bfad-23bb25c553df/kube-rbac-proxy-web/0.log" Jul 09 19:42:27.136771 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.136742 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57ffb4c87-sfxhb_95856664-6f2f-4aec-bfad-23bb25c553df/kube-rbac-proxy/0.log" Jul 09 19:42:27.158279 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.158256 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57ffb4c87-sfxhb_95856664-6f2f-4aec-bfad-23bb25c553df/prom-label-proxy/0.log" Jul 09 19:42:27.180754 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.180729 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57ffb4c87-sfxhb_95856664-6f2f-4aec-bfad-23bb25c553df/kube-rbac-proxy-rules/0.log" Jul 09 19:42:27.203469 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:27.203449 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57ffb4c87-sfxhb_95856664-6f2f-4aec-bfad-23bb25c553df/kube-rbac-proxy-metrics/0.log" Jul 09 19:42:28.521327 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:28.521297 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-btrnl_ee7b42d7-c9a1-4f9c-96ec-53f1b1567a67/networking-console-plugin/0.log" Jul 09 19:42:29.726138 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.726107 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dbb49779f-k7wlh_4fffcff8-0b8e-4605-a31d-00c8cb2f5d0a/console/0.log" Jul 09 19:42:29.772575 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.772536 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n"] Jul 09 19:42:29.772950 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.772937 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73444b66-b3e2-494b-8620-452df7484ad0" containerName="main" Jul 09 19:42:29.772997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.772952 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="73444b66-b3e2-494b-8620-452df7484ad0" containerName="main" Jul 09 19:42:29.772997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.772970 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4391e687-a4b6-4a60-bffe-1e6f6399bc31" containerName="main" Jul 09 19:42:29.772997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.772976 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4391e687-a4b6-4a60-bffe-1e6f6399bc31" containerName="main" Jul 09 19:42:29.773101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.773037 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4391e687-a4b6-4a60-bffe-1e6f6399bc31" containerName="main" Jul 09 19:42:29.773101 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.773047 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="73444b66-b3e2-494b-8620-452df7484ad0" containerName="main" Jul 09 19:42:29.776242 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.776221 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.779582 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.779560 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-797qx\"/\"openshift-service-ca.crt\"" Jul 09 19:42:29.779679 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.779567 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-797qx\"/\"default-dockercfg-7ljkd\"" Jul 09 19:42:29.779679 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.779600 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-797qx\"/\"kube-root-ca.crt\"" Jul 09 19:42:29.785894 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.785869 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n"] Jul 09 19:42:29.856875 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.856845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-sys\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.857033 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.856894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-proc\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.857033 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.856977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5wfz\" (UniqueName: \"kubernetes.io/projected/8996f33f-5dc0-4dec-b16a-3a6112de08be-kube-api-access-h5wfz\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.857033 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.857024 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-lib-modules\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.857142 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.857084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-podres\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958207 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958162 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-proc\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958372 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5wfz\" (UniqueName: \"kubernetes.io/projected/8996f33f-5dc0-4dec-b16a-3a6112de08be-kube-api-access-h5wfz\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958372 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958302 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-proc\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958372 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-lib-modules\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958552 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-podres\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958552 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958408 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-sys\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958552 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958494 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-sys\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958552 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958541 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-lib-modules\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.958730 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.958549 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8996f33f-5dc0-4dec-b16a-3a6112de08be-podres\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:29.966035 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:29.966012 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5wfz\" (UniqueName: \"kubernetes.io/projected/8996f33f-5dc0-4dec-b16a-3a6112de08be-kube-api-access-h5wfz\") pod \"perf-node-gather-daemonset-wfd2n\" (UID: \"8996f33f-5dc0-4dec-b16a-3a6112de08be\") " pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:30.086772 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:30.086739 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:30.232207 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:30.232177 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n"] Jul 09 19:42:30.234362 ip-10-0-133-17 kubenswrapper[2561]: W0709 19:42:30.234324 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8996f33f_5dc0_4dec_b16a_3a6112de08be.slice/crio-f9d6a60665d984461ea3ef8625c3bad66fed8712fbc75f833014ca83404368fc WatchSource:0}: Error finding container f9d6a60665d984461ea3ef8625c3bad66fed8712fbc75f833014ca83404368fc: Status 404 returned error can't find the container with id f9d6a60665d984461ea3ef8625c3bad66fed8712fbc75f833014ca83404368fc Jul 09 19:42:30.828961 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:30.828919 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" event={"ID":"8996f33f-5dc0-4dec-b16a-3a6112de08be","Type":"ContainerStarted","Data":"40bf434c489d8219af32153a1e6600042a4680bcf38b35fe4afc82d72cab4136"} Jul 09 19:42:30.829448 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:30.829021 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:30.829448 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:30.829048 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" event={"ID":"8996f33f-5dc0-4dec-b16a-3a6112de08be","Type":"ContainerStarted","Data":"f9d6a60665d984461ea3ef8625c3bad66fed8712fbc75f833014ca83404368fc"} Jul 09 19:42:30.844977 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:30.844915 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" podStartSLOduration=1.844896882 podStartE2EDuration="1.844896882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:42:30.844594507 +0000 UTC m=+2651.296046969" watchObservedRunningTime="2026-07-09 19:42:30.844896882 +0000 UTC m=+2651.296349340" Jul 09 19:42:31.534905 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:31.534872 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mc95l_8f49887e-0428-4857-8f47-72d48641cea6/dns/0.log" Jul 09 19:42:31.574058 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:31.574027 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mc95l_8f49887e-0428-4857-8f47-72d48641cea6/kube-rbac-proxy/0.log" Jul 09 19:42:31.723997 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:31.723969 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rk9t5_56b19497-c17c-4c4e-85fc-9e1ceedaa60f/dns-node-resolver/0.log" Jul 09 19:42:32.332408 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:32.332376 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2mbf8_080589b7-abe1-4464-a9eb-37eb06937242/node-ca/0.log" Jul 09 19:42:33.422643 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:33.422616 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-db49944d8-gdk2c_e1cdebdb-59c4-4e69-879c-4198a565cc8a/kube-auth-proxy/0.log" Jul 09 19:42:33.556920 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:33.556888 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-cjnxj_8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1/payload-pre-processing/0.log" Jul 09 19:42:33.623367 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:33.623333 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c45df5c76-pqvtd_1b789612-d416-4ae7-880e-71c9bc9939e3/router/0.log" Jul 09 19:42:34.323033 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:34.323004 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dkchf_6736da78-2995-4f4b-b579-543e73b3fbd3/serve-healthcheck-canary/0.log" Jul 09 19:42:34.824814 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:34.824785 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-58bnj_6f57b8ec-116b-428c-a6a9-36300b01abb3/kube-rbac-proxy/0.log" Jul 09 19:42:34.845252 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:34.845226 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-58bnj_6f57b8ec-116b-428c-a6a9-36300b01abb3/exporter/0.log" Jul 09 19:42:34.865705 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:34.865683 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-58bnj_6f57b8ec-116b-428c-a6a9-36300b01abb3/extractor/0.log" Jul 09 19:42:36.845688 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:36.845655 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-797qx/perf-node-gather-daemonset-wfd2n" Jul 09 19:42:37.192556 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:37.192448 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j9q5_48ebe129-5eff-4908-ab95-f01e4e247026/konnectivity-agent/0.log" limit=1024 Jul 09 19:42:37.309411 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:37.309380 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5856874589-nv5m5_1680d844-21aa-4977-9e0c-86e88dc05612/manager/0.log" Jul 09 19:42:37.441080 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:37.441049 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-799d55cdcd-7l5vd_1030534d-6078-49af-9143-88d95a426541/manager/0.log" Jul 09 19:42:37.467659 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:37.467574 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-799d55cdcd-7l5vd_1030534d-6078-49af-9143-88d95a426541/copy-manifests/0.log" Jul 09 19:42:38.801173 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:38.801142 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-868778c6d9-2fzsf_26fddc1b-0e60-4db5-9ebf-558897e45396/manager/0.log" Jul 09 19:42:40.163190 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:40.163111 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-db49944d8-gdk2c_e1cdebdb-59c4-4e69-879c-4198a565cc8a/kube-auth-proxy/0.log" Jul 09 19:42:40.270272 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:40.270235 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-cjnxj_8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1/payload-pre-processing/0.log" Jul 09 19:42:40.310209 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:40.310178 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c45df5c76-pqvtd_1b789612-d416-4ae7-880e-71c9bc9939e3/router/0.log" Jul 09 19:42:41.713070 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:41.713046 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-db49944d8-gdk2c_e1cdebdb-59c4-4e69-879c-4198a565cc8a/kube-auth-proxy/0.log" Jul 09 19:42:41.814607 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:41.814574 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-cjnxj_8ee7cb9f-1301-4302-a3f1-6f4eceaab2d1/payload-pre-processing/0.log" Jul 09 19:42:41.857639 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:41.857614 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c45df5c76-pqvtd_1b789612-d416-4ae7-880e-71c9bc9939e3/router/0.log" Jul 09 19:42:44.468082 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:44.468049 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-wjndj_839a1ce0-a0bd-4a90-873d-91be28e94261/kube-storage-version-migrator-operator/1.log" Jul 09 19:42:44.469140 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:44.469123 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-wjndj_839a1ce0-a0bd-4a90-873d-91be28e94261/kube-storage-version-migrator-operator/0.log" Jul 09 19:42:45.414261 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.414225 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-452qh_6503b59d-776a-448a-899e-2cd7c73ba171/kube-multus/0.log" Jul 09 19:42:45.678303 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.678216 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/kube-multus-additional-cni-plugins/0.log" Jul 09 19:42:45.700022 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.699992 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/egress-router-binary-copy/0.log" Jul 09 19:42:45.723734 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.723699 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/cni-plugins/0.log" Jul 09 19:42:45.744874 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.744850 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/bond-cni-plugin/0.log" Jul 09 19:42:45.771277 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.771250 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/routeoverride-cni/0.log" Jul 09 19:42:45.794576 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.794546 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/whereabouts-cni-bincopy/0.log" Jul 09 19:42:45.818794 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:45.818759 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mdhz8_012fe783-7cfb-4569-85c6-b9133d47bc5d/whereabouts-cni/0.log" Jul 09 19:42:46.082743 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:46.082713 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-stgpg_3f59493d-083f-4990-8df2-09f3c8845e86/network-metrics-daemon/0.log" Jul 09 19:42:46.107713 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:46.107674 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-stgpg_3f59493d-083f-4990-8df2-09f3c8845e86/kube-rbac-proxy/0.log" Jul 09 19:42:47.333074 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.333041 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-controller/0.log" Jul 09 19:42:47.350160 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.350130 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/0.log" Jul 09 19:42:47.350160 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.350136 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovn-acl-logging/1.log" Jul 09 19:42:47.370654 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.370616 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/kube-rbac-proxy-node/0.log" Jul 09 19:42:47.391423 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.391314 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 19:42:47.408108 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.408081 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/northd/0.log" Jul 09 19:42:47.429929 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.429892 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/nbdb/0.log" Jul 09 19:42:47.451110 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.451078 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/sbdb/0.log" Jul 09 19:42:47.505128 ip-10-0-133-17 kubenswrapper[2561]: I0709 19:42:47.505092 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5s_55d0dba2-5711-485d-a7cb-cfbb268bec42/ovnkube-controller/0.log"