Jul 18 01:55:04.338806 ip-10-0-133-113 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 18 01:55:04.338944 ip-10-0-133-113 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 18 01:55:04.339036 ip-10-0-133-113 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 18 01:55:04.339449 ip-10-0-133-113 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 18 01:55:14.572428 ip-10-0-133-113 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 18 01:55:14.572448 ip-10-0-133-113 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8ebf0dadefb2456aba45f529d6a7d86e --
Jul 18 01:57:05.394463 ip-10-0-133-113 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.794039    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800883    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800895    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800898    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800901    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800904    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800907    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800910    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:57:05.842368 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800912    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800915    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800917    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800921    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800923    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800926    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800929    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800931    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800934    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800936    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800939    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800941    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800944    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800952    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800955    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800958    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800960    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800963    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800967    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:57:05.859437 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800971    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800974    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800977    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800979    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800983    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800985    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800988    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800990    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800993    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800996    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.800999    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801001    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801004    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801007    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801009    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801012    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801015    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801017    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801020    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801022    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:57:05.862005 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801025    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801028    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801030    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801032    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801037    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801040    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801043    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801045    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801048    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801050    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801053    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801055    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801058    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801060    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801063    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801066    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801068    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801072    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801076    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:57:05.876526 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801079    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801082    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801085    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801088    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801091    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801093    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801097    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801099    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801102    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801105    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801108    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801110    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801113    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801115    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801117    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801120    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801123    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801125    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801128    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801130    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:57:05.877327 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801133    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801135    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801138    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801140    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801143    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801145    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801148    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801150    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801152    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801155    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801159    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801162    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801164    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801167    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801841    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801848    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801851    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801854    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801858    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801862    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:57:05.878105 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801866    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801869    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801873    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801876    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801879    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801881    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801884    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801886    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801889    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801892    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801895    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801897    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801900    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801902    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801905    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801907    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801910    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801912    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801915    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801917    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:57:05.878874 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801919    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801922    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801925    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801928    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801931    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801934    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801936    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801939    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801942    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801945    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801947    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801950    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801953    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801956    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801958    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801961    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801963    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801966    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801969    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801971    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:57:05.879662 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801973    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801976    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801978    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801981    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801983    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801985    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801988    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801990    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801993    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801996    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.801998    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802001    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802003    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802006    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802008    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802011    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802013    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802016    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802019    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802022    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:57:05.880468 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802024    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802027    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802029    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802032    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802034    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802037    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802040    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802042    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802045    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802048    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802051    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802055    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802057    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802060    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802063    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802065    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802068    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802071    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802073    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802075    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:57:05.881640 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802078    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802080    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802082    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802085    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802087    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802090    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802092    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802095    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802097    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802100    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802103    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802107    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802109    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802474    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802479    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802482    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802485    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802488    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802492    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:57:05.882467 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802495    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802498    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802500    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802503    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802506    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802509    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802514    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802517    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802520    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802523    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802526    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802529    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802532    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802536    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802538    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802541    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802544    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802547    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802549    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802552    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:57:05.883262 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802554    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802557    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802559    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802562    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802564    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802567    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802569    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802572    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802575    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802577    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802580    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802583    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802586    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802589    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802592    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802594    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802597    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802599    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802602    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802605    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:57:05.884055 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802607    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802610    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802612    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802615    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802617    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802619    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802622    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802624    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802627    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802630    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802632    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802635    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802637    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802640    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802642    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802645    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802647    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802649    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802652    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802654    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:57:05.884834 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802657    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802661    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802664    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802666    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802669    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802672    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802674    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802677    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802679    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802682    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802684    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802687    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802690    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802692    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802695    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802699    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802701    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802705    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802707    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:57:05.885654 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802711    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802714    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802717    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802719    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802722    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802725    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802727    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802730    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802733    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802735    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802738    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802740    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802744    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.802747    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802815    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802825    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802831    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802835    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802840    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802843    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 01:57:05.886458 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802846    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802850    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802853    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802856    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802859    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802863    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802866    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802868    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802872    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802875    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802878    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802881    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802885    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802887    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802890    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802893    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802896    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802900    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802902    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802905    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802908    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802911    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802914    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802916    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 01:57:05.887614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802919    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802922    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802927    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802930    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802933    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802936    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802938    2576 flags.go:64] FLAG: --enable-server="true"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802942    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802946    2576 flags.go:64] FLAG: --event-burst="100"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802949    2576 flags.go:64] FLAG: --event-qps="50"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802952    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802955    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802958    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802961    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802964    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802967    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802972    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802975    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802977    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802980    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802983    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802986    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802989    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802991    2576 flags.go:64] FLAG: --feature-gates=""
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802995    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 01:57:05.888438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.802998    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803001    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803004    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803007    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803009    2576 flags.go:64] FLAG: --help="false"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803012    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-133-113.ec2.internal"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803015    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803018    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803021    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803025    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803028    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803031    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803034    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803036    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803039    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803042    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803045    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803048    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803051    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803054    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803057    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803059    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803062    2576 flags.go:64] FLAG: --lock-file=""
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803066    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 01:57:05.889233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803068    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803071    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803077    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803079    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803082    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803085    2576 flags.go:64] FLAG: --logging-format="text"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803088    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803091    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803093    2576 flags.go:64] FLAG: --manifest-url=""
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803096    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803100    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803103    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803106    2576 flags.go:64] FLAG: --max-pods="110"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803109    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803112    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803114    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803117    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803121    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803124    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803127    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803133    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803136    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803139    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 01:57:05.890006 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803142    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803145    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803151    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803153    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803156    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803159    2576 flags.go:64] FLAG: --port="10250"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803162    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803164    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0306385e2a8f8941e"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803167    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803171    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803174    2576 flags.go:64] FLAG: --register-node="true"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803177    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803180    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803183    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803186    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803205    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803209    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803212    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803215    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803218    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803221    2576 flags.go:64] FLAG: --runonce="false"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803223    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803226    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803229    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803232    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 01:57:05.890802 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803235    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803239    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803242    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803245    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803248    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803251    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803254    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803256    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803259    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803263    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803266    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803271    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803274    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803277    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803280    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803283    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803287    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803290    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803293    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803295    2576 flags.go:64] FLAG: --v="2"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803299    2576 flags.go:64] FLAG: --version="false"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803303    2576 flags.go:64] FLAG: --vmodule=""
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803306    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.803309    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803409    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:57:05.891607 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803413    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803417    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803421    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803425    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803428    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803431    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803434    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803437    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803440    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803443    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803446    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803448    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803451    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803454    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803456    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803458    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803461    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803464    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803467    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:57:05.892400 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803471    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803474    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803477    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803480    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803483    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803486    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803488    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803491    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803493    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803496    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803498    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803501    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803503    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803506    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803508    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803511    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803513    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803516    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803519    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803521    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:57:05.992306 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803523    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803527    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803530    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803532    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803535    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803537    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803540    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803542    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803545    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803547    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803550    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803553    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803555    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803557    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803560    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803563    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803566    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803569    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803571    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803574    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:57:05.993099 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803577    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803579    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803582    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803584    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803587    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803589    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803592    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803594    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803596    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803599    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803602    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803604    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803607    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803610    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803613    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803615    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803618    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803620    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803623    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803625    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:57:05.995267 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803628    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803630    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803633    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803635    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803638    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803641    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803643    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803646    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803650    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803652    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803654    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803658    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803661    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803665    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803667    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803670    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803673    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803676    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:57:05.996206 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.803679    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.804442    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.811177    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.811211    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.811227    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.811234    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811305    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811311    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811316    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811320    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811324    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811327    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811331    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811335    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811339    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:57:05.997366 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811343    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811347    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811351    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811355    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811359    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811363    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811367    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811371    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811375    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811379    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811382    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811386    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811389    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811394    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811402    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811405    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811409    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811412    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811416    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811420    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:57:05.998127 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811424    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811428    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811432    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811437    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811442    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811446    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811450    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811454    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811458    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811461    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811465    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811469    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811473    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811477    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811480    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811484    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811489    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811493    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811496    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:57:06.004971 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811499    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811503    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811507    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811510    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811514    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811518    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811524    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811529    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811533    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811536    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811540    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811544    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811548    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811552    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811556    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811559    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811563    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811568    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811571    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811575    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:57:06.005750 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811580    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811585    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811589    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811594    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811599    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811605    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811609    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811613    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811618    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811623    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811627    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811631    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811635    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811639    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811642    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811646    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811650    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811653    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811659    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:57:06.007307 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811664    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811668    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811672    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811675    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811678    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811682    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811685    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811689    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811693    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811697    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811701    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811705    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.811711    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811862    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811869    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:57:06.008091 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811874    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811877    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811881    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811885    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811888    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811892    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811896    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811899    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811904    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811908    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811912    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811915    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811919    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811924    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811927    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811931    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811935    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811941    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811945    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811949    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:57:06.008945 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811952    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811956    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811960    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811963    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811966    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811970    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811974    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811978    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811981    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811984    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811987    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811991    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811995    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.811999    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812003    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812006    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812011    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812016    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812020    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812024    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:57:06.112840 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812027    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812031    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812035    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812038    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812041    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812045    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812049    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812052    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812056    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812062    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812066    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812069    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812074    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812077    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812081    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812085    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812088    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812091    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812095    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812099    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:57:06.113615 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812103    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812106    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812109    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812113    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812116    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812120    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812124    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812127    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812131    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812135    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812138    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812142    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812146    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812149    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812153    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812157    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812160    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812164    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812167    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812171    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:57:06.114384 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812175    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812181    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812184    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812201    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812206    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812210    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812214    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812217    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812221    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812225    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812228    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812234    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812239    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812245    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812249    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812254    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:57:06.116351 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:05.812258    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.812265    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.812376    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.815892    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.817079    2576 server.go:1019] "Starting client certificate rotation"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.817177    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.817221    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.819769    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.820895    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.831951    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.834727    2576 log.go:25] "Validated CRI v1 image API"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.835521    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 01:57:06.117126 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.837850    2576 fs.go:135] Filesystem UUIDs: map[0caf4c76-1853-45ed-b67e-fa4d97932e8a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e07e04f7-fb47-4c1b-9def-81fb4a5b8aeb:/dev/nvme0n1p3]
Jul 18 01:57:06.117926 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.837869    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.844206    2576 manager.go:216] Machine: {Timestamp:2026-07-18 01:57:05.842682471 +0000 UTC m=+0.335326288 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100634 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b7b666a7800290f37d4ee69117831 SystemUUID:ec2b7b66-6a78-0029-0f37-d4ee69117831 BootID:8ebf0dad-efb2-456a-ba45-f529d6a7d86e Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:ca:2b:ef:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:ca:2b:ef:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:7d:96:e8:47:31 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.844298    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.844365    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.846475    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.846504    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-113.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.847219    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.847230    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 01:57:06.120324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.847237    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.847521    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.848463    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.849679    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853625    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853778    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853809    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853821    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853835    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853843    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.853855    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.854304    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:57:06.121117 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:05.854319    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.186451    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mmx5m"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.186534    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mmx5m"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.186769    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.190372    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.190401    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191478    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191492    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191498    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191503    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191512    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191517    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191525    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191530    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191536    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191542    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191551    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.191556    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.192372    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-113.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.192413    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.192423    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.192470    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.195875    2576 server.go:1268] "Started kubelet"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.196215    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.196538    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.196597    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.197634    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.198560    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.202948    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.203809    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.205183    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.205282    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.205752    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.205754    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.205774    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.205945    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.205955    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.206171    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206296    2576 factory.go:55] Registering systemd factory
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206307    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206528    2576 factory.go:153] Registering CRI-O factory
Jul 18 01:57:06.229542 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206541    2576 factory.go:223] Registration of the crio container factory successfully
Jul 18 01:57:06.196466 ip-10-0-133-113 systemd[1]: Started Kubernetes Kubelet.
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206581    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206609    2576 factory.go:103] Registering Raw factory
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.206621    2576 manager.go:1196] Started watching for new ooms in manager
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.207030    2576 manager.go:349] Starting recovery of all containers
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.207566    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.210853    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-113.ec2.internal\" not found" node="ip-10-0-133-113.ec2.internal"
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.225559    2576 manager.go:354] Recovery completed
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.225591    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.231336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.229369    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:06.234523 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234358    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:06.234523 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234386    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:06.234523 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234401    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:06.235027 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234904    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 01:57:06.235027 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234915    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:06.235027 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234914    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 01:57:06.235027 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.234954    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.235605    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.235628    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.235642    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.237234    2576 policy_none.go:49] "None policy: Start"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.237253    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.237267    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.239227    2576 policy_none.go:47] "Start"
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.286257    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.367691 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.306371    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.483560 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.406906    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.600588 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.507082    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.600588 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.550859    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.600588 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.550900    2576 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.600588 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.586904    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.604304 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.603258    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.609476 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.607451    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.707685 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.662564    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:06.710622 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.707755    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.823222 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.808000    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:06.823222 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.817121    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 01:57:06.823222 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.817239    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:57:06.926138 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.908582    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.025073 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:06.929858    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:07.025073 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:06.929872    2576 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-113.ec2.internal" not found
Jul 18 01:57:07.025073 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.009089    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.070240 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070226    2576 manager.go:337] "Starting Device Plugin manager"
Jul 18 01:57:07.070318 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.070264    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 01:57:07.070318 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070275    2576 server.go:87] "Starting device plugin registration server"
Jul 18 01:57:07.070551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070539    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 01:57:07.070607 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070554    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 01:57:07.070648 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070620    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 01:57:07.070723 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070702    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 01:57:07.070723 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.070714    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 01:57:07.071268 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.071244    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 01:57:07.071339 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.071298    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.072866 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.072841    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 01:57:07.074156 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.074133    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 01:57:07.074256 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.074172    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 01:57:07.074256 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.074214    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 01:57:07.074256 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.074251    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 01:57:07.076267 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.076250    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:57:07.084411 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.084396    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:57:07.171551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.171535    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:07.172294 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.172278    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:07.172356 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.172308    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:07.172356 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.172329    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:07.172410 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.172371    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.174424 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.174403    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"]
Jul 18 01:57:07.174468 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.174460    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:07.175097 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.175082    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:07.175161 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.175106    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:07.175161 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.175116    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:07.176371 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.176359    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:07.176484 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.176471    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.176513 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.176500    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:07.176970 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.176955    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:07.177045 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.176980    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:07.177045 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.176992    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:07.177892 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.177877    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:07.177961 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.177902    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:07.177961 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.177913    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:07.179072 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.179056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.179163 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.179086    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:57:07.179775 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.179759    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:57:07.179851 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.179787    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:57:07.179851 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.179800    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:57:07.182394 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.182378    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.182450 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.182397    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-113.ec2.internal\": node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.192025 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.192006    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:52:05 +0000 UTC" deadline="2028-04-02 13:58:38.193531124 +0000 UTC"
Jul 18 01:57:07.192079 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.192024    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14988h1m31.001508998s"
Jul 18 01:57:07.195341 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.195324    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.203656 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.203637    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-113.ec2.internal\" not found" node="ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.207739 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.207726    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-113.ec2.internal\" not found" node="ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.209765 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.209750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a31f88c21c7864a4b5269fc72fabd5c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal\" (UID: \"a31f88c21c7864a4b5269fc72fabd5c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.209819 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.209772    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a31f88c21c7864a4b5269fc72fabd5c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal\" (UID: \"a31f88c21c7864a4b5269fc72fabd5c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.209819 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.209791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1303fa5083c2705ceafaaf4e41a3f21-config\") pod \"kube-apiserver-proxy-ip-10-0-133-113.ec2.internal\" (UID: \"a1303fa5083c2705ceafaaf4e41a3f21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.295779 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.295732    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.310532 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.310512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a31f88c21c7864a4b5269fc72fabd5c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal\" (UID: \"a31f88c21c7864a4b5269fc72fabd5c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.310613 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.310539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a31f88c21c7864a4b5269fc72fabd5c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal\" (UID: \"a31f88c21c7864a4b5269fc72fabd5c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.310613 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.310560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1303fa5083c2705ceafaaf4e41a3f21-config\") pod \"kube-apiserver-proxy-ip-10-0-133-113.ec2.internal\" (UID: \"a1303fa5083c2705ceafaaf4e41a3f21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.310613 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.310598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a31f88c21c7864a4b5269fc72fabd5c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal\" (UID: \"a31f88c21c7864a4b5269fc72fabd5c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.310718 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.310601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1303fa5083c2705ceafaaf4e41a3f21-config\") pod \"kube-apiserver-proxy-ip-10-0-133-113.ec2.internal\" (UID: \"a1303fa5083c2705ceafaaf4e41a3f21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.310718 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.310652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a31f88c21c7864a4b5269fc72fabd5c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal\" (UID: \"a31f88c21c7864a4b5269fc72fabd5c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.396297 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.396272    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.496594 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.496576    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.506786 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.506764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.510557 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.510516    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.596718 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.596672    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.697256 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.697236    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-113.ec2.internal\" not found"
Jul 18 01:57:07.760869 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.760849    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:57:07.805736 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.805716    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.814837 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.814819    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:57:07.815830 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.815817    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal"
Jul 18 01:57:07.825131 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.825111    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:57:07.855264 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.855220    2576 apiserver.go:52] "Watching apiserver"
Jul 18 01:57:07.862513 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.862493    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 01:57:07.862853 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.862834    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-l87cg","openshift-ovn-kubernetes/ovnkube-node-2s9ds","kube-system/global-pull-secret-syncer-wpn87","kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal","openshift-cluster-node-tuning-operator/tuned-k87w2","openshift-dns/node-resolver-ml5cr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal","openshift-multus/multus-additional-cni-plugins-d8rc6","openshift-multus/network-metrics-daemon-tnrz7","kube-system/konnectivity-agent-fd7cq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5","openshift-image-registry/node-ca-fdb4x","openshift-multus/multus-nffmj","openshift-network-diagnostics/network-check-target-wqbsp"]
Jul 18 01:57:07.864548 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.864525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:07.864639 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.864596    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:07.866636 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.866618    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.868204 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.868172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:07.868305 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.868253    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:07.869406 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.869388    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 01:57:07.869795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.869773    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:07.870446 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.870426    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 01:57:07.870557 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.870446    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 01:57:07.870557 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.870533    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.870656 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.870576    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.870656 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.870575    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 01:57:07.870656 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.870624    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t7grl\""
Jul 18 01:57:07.871416 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.871402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.871942 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.871924    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.872028 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.871954    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 01:57:07.872250 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.872234    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4jb2b\""
Jul 18 01:57:07.872333 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.872239    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.872980 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.872967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:07.873376 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.873352    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5nr9r\""
Jul 18 01:57:07.873484 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.873419    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.873484 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.873439    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.874713 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.874698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.875256 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.875241    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 01:57:07.875336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.875273    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.875437 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.875425    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.875687 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.875675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jhg59\""
Jul 18 01:57:07.876382 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.876367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:07.876914 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.876898    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 01:57:07.877029 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.877011    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.877165 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.877147    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.877243 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.877148    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqh5h\""
Jul 18 01:57:07.878253 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.878238    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.878487 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.878384    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 01:57:07.878635 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.878620    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h7jnw\""
Jul 18 01:57:07.878698 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.878683    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 01:57:07.881795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.880594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:07.881795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.881222    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.881795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.881629    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 01:57:07.881988 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.881893    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 01:57:07.882041 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.881992    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2l9g5\""
Jul 18 01:57:07.882350 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.882332    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.882579 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.882558    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 01:57:07.883478 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.883459    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 01:57:07.884160 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.883891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.884160 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.884094    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 01:57:07.884160 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.884160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l9q98\""
Jul 18 01:57:07.886253 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.885982    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 01:57:07.886323 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.886307    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m8wvx\""
Jul 18 01:57:07.887096 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.887076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:07.887203 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:07.887132    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:07.907385 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.907364    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 01:57:07.912996 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.912980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-cnibin\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913071 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-netns\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913071 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-daemon-config\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913071 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913039    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysctl-conf\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913218 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-tuned\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913218 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-system-cni-dir\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.913218 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-cni-bin\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.913218 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-etc-selinux\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-conf-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-host\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-systemd-units\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6887c15-1d53-4eae-91ac-f130359d129d-host-slash\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-kubelet-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-kubelet\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-etc-kubernetes\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysconfig\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-sys\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d780625-cebc-417b-9f06-1f00ec4f972b-host\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-systemd\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-cni-binary-copy\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-systemd\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-run\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-var-lib-kubelet\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d780625-cebc-417b-9f06-1f00ec4f972b-serviceca\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913682    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcsfw\" (UniqueName: \"kubernetes.io/projected/91dea430-55c6-42e2-9589-148379a893aa-kube-api-access-gcsfw\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-socket-dir-parent\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913733 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-hostroot\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-lib-modules\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-cnibin\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-os-release\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm8bp\" (UniqueName: \"kubernetes.io/projected/678a9215-5982-4c4c-82d2-bc8262c8ab30-kube-api-access-nm8bp\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-log-socket\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ffr\" (UniqueName: \"kubernetes.io/projected/a6887c15-1d53-4eae-91ac-f130359d129d-kube-api-access-n2ffr\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-cni-bin\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b506e3e4-9db5-4d4f-86f4-88cca7974d24-dbus\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-modprobe-d\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.913956 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-socket-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.913995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wfl8\" (UniqueName: \"kubernetes.io/projected/af19b7ac-2be2-4c32-84bc-3b1793db1333-kube-api-access-6wfl8\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914019    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/294352e3-ce7b-4d1b-a742-1ed861819583-agent-certs\") pod \"konnectivity-agent-fd7cq\" (UID: \"294352e3-ce7b-4d1b-a742-1ed861819583\") " pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-tmp\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-kubelet\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-slash\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b506e3e4-9db5-4d4f-86f4-88cca7974d24-kubelet-config\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-run-netns\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86bbt\" (UniqueName: \"kubernetes.io/projected/80423d99-c464-4827-883e-a5be2a3cddd7-kube-api-access-86bbt\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-kubernetes\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.914372 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914337    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgg7l\" (UniqueName: \"kubernetes.io/projected/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-kube-api-access-bgg7l\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8bsm\" (UniqueName: \"kubernetes.io/projected/1d780625-cebc-417b-9f06-1f00ec4f972b-kube-api-access-r8bsm\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-ovnkube-config\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-device-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-sys-fs\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91dea430-55c6-42e2-9589-148379a893aa-hosts-file\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-node-log\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-multus-certs\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914577    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-ovn\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-registration-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:07.914907 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-cni-netd\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-os-release\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914682    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysctl-d\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-etc-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914714    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-env-overrides\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6887c15-1d53-4eae-91ac-f130359d129d-iptables-alerter-script\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914765    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-k8s-cni-cncf-io\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8h6\" (UniqueName: \"kubernetes.io/projected/db62ea2d-a460-48d1-b3ea-16278eff9369-kube-api-access-6p8h6\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-system-cni-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-cni-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-cni-multus\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-cni-binary-copy\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91dea430-55c6-42e2-9589-148379a893aa-tmp-dir\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:07.915404 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.914977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-var-lib-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.915743 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.915002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/294352e3-ce7b-4d1b-a742-1ed861819583-konnectivity-ca\") pod \"konnectivity-agent-fd7cq\" (UID: \"294352e3-ce7b-4d1b-a742-1ed861819583\") " pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:07.915743 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.915024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jdkw\" (UniqueName: \"kubernetes.io/projected/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-kube-api-access-7jdkw\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:07.915743 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.915044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80423d99-c464-4827-883e-a5be2a3cddd7-ovn-node-metrics-cert\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:07.915743 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:07.915065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-ovnkube-script-lib\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.015681 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-cnibin\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-netns\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-daemon-config\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysctl-conf\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015728    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-tuned\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-cnibin\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-system-cni-dir\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-cni-bin\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-etc-selinux\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-conf-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysctl-conf\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-host\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-system-cni-dir\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-etc-selinux\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-systemd-units\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.016112 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-conf-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015914    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-host\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-cni-bin\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6887c15-1d53-4eae-91ac-f130359d129d-host-slash\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015938    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-systemd-units\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-kubelet-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-netns\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-kubelet-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.015995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6887c15-1d53-4eae-91ac-f130359d129d-host-slash\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-kubelet\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-etc-kubernetes\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-kubelet\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysconfig\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016075    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysconfig\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.016795 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-sys\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d780625-cebc-417b-9f06-1f00ec4f972b-host\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-etc-kubernetes\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016146    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d780625-cebc-417b-9f06-1f00ec4f972b-host\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-systemd\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-sys\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016215    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-cni-binary-copy\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-systemd\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-systemd\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-run\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-systemd\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-run\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-var-lib-kubelet\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d780625-cebc-417b-9f06-1f00ec4f972b-serviceca\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.017407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-var-lib-kubelet\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-daemon-config\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcsfw\" (UniqueName: \"kubernetes.io/projected/91dea430-55c6-42e2-9589-148379a893aa-kube-api-access-gcsfw\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-socket-dir-parent\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-hostroot\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-lib-modules\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-cnibin\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-os-release\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm8bp\" (UniqueName: \"kubernetes.io/projected/678a9215-5982-4c4c-82d2-bc8262c8ab30-kube-api-access-nm8bp\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-log-socket\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ffr\" (UniqueName: \"kubernetes.io/projected/a6887c15-1d53-4eae-91ac-f130359d129d-kube-api-access-n2ffr\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-cni-bin\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.018022 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-cni-binary-copy\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-hostroot\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-os-release\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b506e3e4-9db5-4d4f-86f4-88cca7974d24-dbus\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-modprobe-d\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016756    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-socket-dir-parent\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-socket-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-log-socket\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wfl8\" (UniqueName: \"kubernetes.io/projected/af19b7ac-2be2-4c32-84bc-3b1793db1333-kube-api-access-6wfl8\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/294352e3-ce7b-4d1b-a742-1ed861819583-agent-certs\") pod \"konnectivity-agent-fd7cq\" (UID: \"294352e3-ce7b-4d1b-a742-1ed861819583\") " pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d780625-cebc-417b-9f06-1f00ec4f972b-serviceca\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-cnibin\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.018638 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-tmp\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-cni-bin\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-kubelet\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-slash\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016958    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-socket-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b506e3e4-9db5-4d4f-86f4-88cca7974d24-kubelet-config\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.016994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-modprobe-d\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-kubelet\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017069    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-run-netns\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86bbt\" (UniqueName: \"kubernetes.io/projected/80423d99-c464-4827-883e-a5be2a3cddd7-kube-api-access-86bbt\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-lib-modules\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017129    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b506e3e4-9db5-4d4f-86f4-88cca7974d24-dbus\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:08.019281 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.017160    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-kubernetes\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgg7l\" (UniqueName: \"kubernetes.io/projected/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-kube-api-access-bgg7l\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b506e3e4-9db5-4d4f-86f4-88cca7974d24-kubelet-config\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.017258    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:08.517238195 +0000 UTC m=+3.009882003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8bsm\" (UniqueName: \"kubernetes.io/projected/1d780625-cebc-417b-9f06-1f00ec4f972b-kube-api-access-r8bsm\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-run-netns\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-ovnkube-config\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-kubernetes\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-device-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-sys-fs\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.019829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91dea430-55c6-42e2-9589-148379a893aa-hosts-file\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017589    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-node-log\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-sys-fs\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-multus-certs\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-device-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-slash\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91dea430-55c6-42e2-9589-148379a893aa-hosts-file\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-ovn\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-node-log\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-registration-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.017842    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-multus-certs\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.020236 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-run-ovn\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.017907    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:08.517892491 +0000 UTC m=+3.010536309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-cni-netd\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/af19b7ac-2be2-4c32-84bc-3b1793db1333-registration-dir\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.017966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-host-cni-netd\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-os-release\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysctl-d\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-os-release\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-ovnkube-config\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-etc-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-env-overrides\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6887c15-1d53-4eae-91ac-f130359d129d-iptables-alerter-script\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-etc-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-sysctl-d\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.020753 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-k8s-cni-cncf-io\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-run-k8s-cni-cncf-io\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8h6\" (UniqueName: \"kubernetes.io/projected/db62ea2d-a460-48d1-b3ea-16278eff9369-kube-api-access-6p8h6\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-system-cni-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-system-cni-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-cni-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-cni-multus\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-cni-binary-copy\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-multus-cni-dir\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91dea430-55c6-42e2-9589-148379a893aa-tmp-dir\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-host-var-lib-cni-multus\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-env-overrides\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-var-lib-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/294352e3-ce7b-4d1b-a742-1ed861819583-konnectivity-ca\") pod \"konnectivity-agent-fd7cq\" (UID: \"294352e3-ce7b-4d1b-a742-1ed861819583\") " pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:08.021415 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdkw\" (UniqueName: \"kubernetes.io/projected/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-kube-api-access-7jdkw\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80423d99-c464-4827-883e-a5be2a3cddd7-ovn-node-metrics-cert\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018628    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-ovnkube-script-lib\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db62ea2d-a460-48d1-b3ea-16278eff9369-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6887c15-1d53-4eae-91ac-f130359d129d-iptables-alerter-script\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db62ea2d-a460-48d1-b3ea-16278eff9369-cni-binary-copy\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91dea430-55c6-42e2-9589-148379a893aa-tmp-dir\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.018999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80423d99-c464-4827-883e-a5be2a3cddd7-var-lib-openvswitch\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.019140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80423d99-c464-4827-883e-a5be2a3cddd7-ovnkube-script-lib\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.019158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/294352e3-ce7b-4d1b-a742-1ed861819583-konnectivity-ca\") pod \"konnectivity-agent-fd7cq\" (UID: \"294352e3-ce7b-4d1b-a742-1ed861819583\") " pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.019444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-tmp\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.019611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/294352e3-ce7b-4d1b-a742-1ed861819583-agent-certs\") pod \"konnectivity-agent-fd7cq\" (UID: \"294352e3-ce7b-4d1b-a742-1ed861819583\") " pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.019664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-etc-tuned\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.021895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.020662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80423d99-c464-4827-883e-a5be2a3cddd7-ovn-node-metrics-cert\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.024342 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.024111    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:08.024342 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.024133    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:08.024342 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.024153    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:08.024342 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.024224    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:57:08.524207043 +0000 UTC m=+3.016850851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:08.025848 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.025795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wfl8\" (UniqueName: \"kubernetes.io/projected/af19b7ac-2be2-4c32-84bc-3b1793db1333-kube-api-access-6wfl8\") pod \"aws-ebs-csi-driver-node-97wx5\" (UID: \"af19b7ac-2be2-4c32-84bc-3b1793db1333\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.025930 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.025875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgg7l\" (UniqueName: \"kubernetes.io/projected/9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05-kube-api-access-bgg7l\") pod \"tuned-k87w2\" (UID: \"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05\") " pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.026605 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.026582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ffr\" (UniqueName: \"kubernetes.io/projected/a6887c15-1d53-4eae-91ac-f130359d129d-kube-api-access-n2ffr\") pod \"iptables-alerter-l87cg\" (UID: \"a6887c15-1d53-4eae-91ac-f130359d129d\") " pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.026912 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.026886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8bsm\" (UniqueName: \"kubernetes.io/projected/1d780625-cebc-417b-9f06-1f00ec4f972b-kube-api-access-r8bsm\") pod \"node-ca-fdb4x\" (UID: \"1d780625-cebc-417b-9f06-1f00ec4f972b\") " pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.026912 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.026906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcsfw\" (UniqueName: \"kubernetes.io/projected/91dea430-55c6-42e2-9589-148379a893aa-kube-api-access-gcsfw\") pod \"node-resolver-ml5cr\" (UID: \"91dea430-55c6-42e2-9589-148379a893aa\") " pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.027054 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.026986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm8bp\" (UniqueName: \"kubernetes.io/projected/678a9215-5982-4c4c-82d2-bc8262c8ab30-kube-api-access-nm8bp\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:08.027527 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.027509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86bbt\" (UniqueName: \"kubernetes.io/projected/80423d99-c464-4827-883e-a5be2a3cddd7-kube-api-access-86bbt\") pod \"ovnkube-node-2s9ds\" (UID: \"80423d99-c464-4827-883e-a5be2a3cddd7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.027896 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.027879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdkw\" (UniqueName: \"kubernetes.io/projected/b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9-kube-api-access-7jdkw\") pod \"multus-nffmj\" (UID: \"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9\") " pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.027960 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.027922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8h6\" (UniqueName: \"kubernetes.io/projected/db62ea2d-a460-48d1-b3ea-16278eff9369-kube-api-access-6p8h6\") pod \"multus-additional-cni-plugins-d8rc6\" (UID: \"db62ea2d-a460-48d1-b3ea-16278eff9369\") " pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.073242 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.073217    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31f88c21c7864a4b5269fc72fabd5c5.slice/crio-8f1656f0a917d3f9d83f07321e74d1beec2dc33482a3233d57081e2f4284c03f WatchSource:0}: Error finding container 8f1656f0a917d3f9d83f07321e74d1beec2dc33482a3233d57081e2f4284c03f: Status 404 returned error can't find the container with id 8f1656f0a917d3f9d83f07321e74d1beec2dc33482a3233d57081e2f4284c03f
Jul 18 01:57:08.077316 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.077285    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal" event={"ID":"a31f88c21c7864a4b5269fc72fabd5c5","Type":"ContainerStarted","Data":"8f1656f0a917d3f9d83f07321e74d1beec2dc33482a3233d57081e2f4284c03f"}
Jul 18 01:57:08.078593 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.078580    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:57:08.082114 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.082087    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:57:08.085043 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.085026    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1303fa5083c2705ceafaaf4e41a3f21.slice/crio-a96eded590bbcbaceb97da47b303dbc796f798eef1a2de241bbfcbedf8d0479a WatchSource:0}: Error finding container a96eded590bbcbaceb97da47b303dbc796f798eef1a2de241bbfcbedf8d0479a: Status 404 returned error can't find the container with id a96eded590bbcbaceb97da47b303dbc796f798eef1a2de241bbfcbedf8d0479a
Jul 18 01:57:08.190573 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.190528    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:08.198346 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.198328    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80423d99_c464_4827_883e_a5be2a3cddd7.slice/crio-a533a4258388f175545672979d0102a6e955a9edd38825e6f16e32a074a104a7 WatchSource:0}: Error finding container a533a4258388f175545672979d0102a6e955a9edd38825e6f16e32a074a104a7: Status 404 returned error can't find the container with id a533a4258388f175545672979d0102a6e955a9edd38825e6f16e32a074a104a7
Jul 18 01:57:08.206336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.206322    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 01:57:08.215152 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.215137    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:57:08.218924 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.218910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l87cg"
Jul 18 01:57:08.226029 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.226011    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6887c15_1d53_4eae_91ac_f130359d129d.slice/crio-99b6e92027606bb68eb815304b26f38497b3088455afb8be7bad7c065af1706d WatchSource:0}: Error finding container 99b6e92027606bb68eb815304b26f38497b3088455afb8be7bad7c065af1706d: Status 404 returned error can't find the container with id 99b6e92027606bb68eb815304b26f38497b3088455afb8be7bad7c065af1706d
Jul 18 01:57:08.237635 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.237618    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k87w2"
Jul 18 01:57:08.237947 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.237933    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qphwm"
Jul 18 01:57:08.243090 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.243072    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qphwm"
Jul 18 01:57:08.244729 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.244712    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8eb3d7_3e5d_4fef_bc3f_4ee401fcfe05.slice/crio-562428963bdb14851584abdcfb59826149bdc11c8a19919fbeaecfc4a3e0e985 WatchSource:0}: Error finding container 562428963bdb14851584abdcfb59826149bdc11c8a19919fbeaecfc4a3e0e985: Status 404 returned error can't find the container with id 562428963bdb14851584abdcfb59826149bdc11c8a19919fbeaecfc4a3e0e985
Jul 18 01:57:08.253334 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.253316    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdb4x"
Jul 18 01:57:08.256626 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.256608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5"
Jul 18 01:57:08.260495 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.260478    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d780625_cebc_417b_9f06_1f00ec4f972b.slice/crio-e4c71be06f400c127ab2bddbae00a33ad2008c4931acf34feebfe4a581ff39e5 WatchSource:0}: Error finding container e4c71be06f400c127ab2bddbae00a33ad2008c4931acf34feebfe4a581ff39e5: Status 404 returned error can't find the container with id e4c71be06f400c127ab2bddbae00a33ad2008c4931acf34feebfe4a581ff39e5
Jul 18 01:57:08.265179 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.265163    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf19b7ac_2be2_4c32_84bc_3b1793db1333.slice/crio-b0497e83dc1391e18ac89697d74c6323140810d70211cf631ee3a48c771603e8 WatchSource:0}: Error finding container b0497e83dc1391e18ac89697d74c6323140810d70211cf631ee3a48c771603e8: Status 404 returned error can't find the container with id b0497e83dc1391e18ac89697d74c6323140810d70211cf631ee3a48c771603e8
Jul 18 01:57:08.285760 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.285738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:08.290178 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.290165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d8rc6"
Jul 18 01:57:08.294383 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.294365    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294352e3_ce7b_4d1b_a742_1ed861819583.slice/crio-314af1b1c5c8a2cf56527aef9f6091a8ac8e810a60ec212887676aa0f2894917 WatchSource:0}: Error finding container 314af1b1c5c8a2cf56527aef9f6091a8ac8e810a60ec212887676aa0f2894917: Status 404 returned error can't find the container with id 314af1b1c5c8a2cf56527aef9f6091a8ac8e810a60ec212887676aa0f2894917
Jul 18 01:57:08.297869 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.297852    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb62ea2d_a460_48d1_b3ea_16278eff9369.slice/crio-da6505c0709ed50e0389b5757a7e6c61f393149a0b6f4bf2ac973d1fbaf4485a WatchSource:0}: Error finding container da6505c0709ed50e0389b5757a7e6c61f393149a0b6f4bf2ac973d1fbaf4485a: Status 404 returned error can't find the container with id da6505c0709ed50e0389b5757a7e6c61f393149a0b6f4bf2ac973d1fbaf4485a
Jul 18 01:57:08.302819 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.302807    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ml5cr"
Jul 18 01:57:08.308075 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.308057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nffmj"
Jul 18 01:57:08.313462 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.313445    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91dea430_55c6_42e2_9589_148379a893aa.slice/crio-15b79e5d1fd70f68b9b22744d049adcd64e1867e6bc61044c57cff91c7b1aa97 WatchSource:0}: Error finding container 15b79e5d1fd70f68b9b22744d049adcd64e1867e6bc61044c57cff91c7b1aa97: Status 404 returned error can't find the container with id 15b79e5d1fd70f68b9b22744d049adcd64e1867e6bc61044c57cff91c7b1aa97
Jul 18 01:57:08.317070 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:57:08.317055    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07ceb7e_d2d3_4bf7_8fa4_abb616ffb2f9.slice/crio-a24a66b14ff29532aa907b3d27456f251af28767cf5c6d6b11059de6c9d64282 WatchSource:0}: Error finding container a24a66b14ff29532aa907b3d27456f251af28767cf5c6d6b11059de6c9d64282: Status 404 returned error can't find the container with id a24a66b14ff29532aa907b3d27456f251af28767cf5c6d6b11059de6c9d64282
Jul 18 01:57:08.523235 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.523207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:08.523377 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.523300    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:08.523447 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.523432    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:08.523504 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.523469    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:08.523584 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.523480    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:09.52346819 +0000 UTC m=+4.016111995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:08.523650 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.523601    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:09.523584221 +0000 UTC m=+4.016228034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:08.623903 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:08.623872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:08.624045 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.624035    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:08.624121 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.624053    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:08.624121 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.624064    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:08.624121 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:08.624113    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:57:09.624095708 +0000 UTC m=+4.116739527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:09.074999 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.074971    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:09.075643 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.075088    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:09.075643 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.075459    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:09.075643 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.075549    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:09.075643 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.075605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:09.075831 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.075666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:09.100872 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.100844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal" event={"ID":"a1303fa5083c2705ceafaaf4e41a3f21","Type":"ContainerStarted","Data":"a96eded590bbcbaceb97da47b303dbc796f798eef1a2de241bbfcbedf8d0479a"}
Jul 18 01:57:09.123731 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.123702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ml5cr" event={"ID":"91dea430-55c6-42e2-9589-148379a893aa","Type":"ContainerStarted","Data":"15b79e5d1fd70f68b9b22744d049adcd64e1867e6bc61044c57cff91c7b1aa97"}
Jul 18 01:57:09.136936 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.136910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5" event={"ID":"af19b7ac-2be2-4c32-84bc-3b1793db1333","Type":"ContainerStarted","Data":"b0497e83dc1391e18ac89697d74c6323140810d70211cf631ee3a48c771603e8"}
Jul 18 01:57:09.157636 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.157608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l87cg" event={"ID":"a6887c15-1d53-4eae-91ac-f130359d129d","Type":"ContainerStarted","Data":"99b6e92027606bb68eb815304b26f38497b3088455afb8be7bad7c065af1706d"}
Jul 18 01:57:09.174045 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.174016    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"a533a4258388f175545672979d0102a6e955a9edd38825e6f16e32a074a104a7"}
Jul 18 01:57:09.192376 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.192353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nffmj" event={"ID":"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9","Type":"ContainerStarted","Data":"a24a66b14ff29532aa907b3d27456f251af28767cf5c6d6b11059de6c9d64282"}
Jul 18 01:57:09.210416 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.210393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerStarted","Data":"da6505c0709ed50e0389b5757a7e6c61f393149a0b6f4bf2ac973d1fbaf4485a"}
Jul 18 01:57:09.228205 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.228157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fd7cq" event={"ID":"294352e3-ce7b-4d1b-a742-1ed861819583","Type":"ContainerStarted","Data":"314af1b1c5c8a2cf56527aef9f6091a8ac8e810a60ec212887676aa0f2894917"}
Jul 18 01:57:09.244293 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.244273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdb4x" event={"ID":"1d780625-cebc-417b-9f06-1f00ec4f972b","Type":"ContainerStarted","Data":"e4c71be06f400c127ab2bddbae00a33ad2008c4931acf34feebfe4a581ff39e5"}
Jul 18 01:57:09.246455 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.246269    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:52:08 +0000 UTC" deadline="2028-03-31 02:08:05.697436768 +0000 UTC"
Jul 18 01:57:09.246455 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.246296    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14928h10m56.45114533s"
Jul 18 01:57:09.263700 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.263665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k87w2" event={"ID":"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05","Type":"ContainerStarted","Data":"562428963bdb14851584abdcfb59826149bdc11c8a19919fbeaecfc4a3e0e985"}
Jul 18 01:57:09.530202 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.530148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:09.530365 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.530260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:09.530423 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.530407    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:09.530469 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.530460    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:11.530444027 +0000 UTC m=+6.023087832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:09.530811 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.530794    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:09.530887 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.530841    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:11.530828291 +0000 UTC m=+6.023472100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:09.630964 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:09.630929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:09.631116 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.631097    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:09.631210 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.631118    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:09.631210 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.631131    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:09.631210 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:09.631200    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:57:11.631170468 +0000 UTC m=+6.123814274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:10.247273 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:10.247203    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:52:08 +0000 UTC" deadline="2028-03-02 15:28:37.558808762 +0000 UTC"
Jul 18 01:57:10.247273 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:10.247241    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14245h31m27.311571511s"
Jul 18 01:57:11.075324 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:11.075289    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:11.075515 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:11.075337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:11.075515 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:11.075306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:11.075515 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.075448    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:11.075673 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.075576    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:11.075673 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.075647    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:11.551269 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:11.551232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:11.551695 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:11.551431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:11.551695 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.551561    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:11.551695 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.551611    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:15.551596626 +0000 UTC m=+10.044240448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:11.552150 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.551991    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:11.552416 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.552352    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:15.552029386 +0000 UTC m=+10.044673194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:11.653339 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:11.652685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:11.653339 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.652916    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:11.653339 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.652937    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:11.653339 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.652949    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:11.653339 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:11.653002    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:57:15.652984992 +0000 UTC m=+10.145628799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:13.079846 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:13.079814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:13.080419 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:13.079818    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:13.080419 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:13.079957    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:13.080419 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:13.080050    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:13.080419 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:13.079823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:13.080419 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:13.080151    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:15.075180 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:15.075133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:15.075541 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:15.075132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:15.075541 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:15.075150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:15.075541 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.075277    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:15.075715 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.075689    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:15.075799 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.075780    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:15.585470 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:15.585429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:15.585703 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:15.585520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:15.585703 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.585637    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:15.585703 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.585701    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:23.585681358 +0000 UTC m=+18.078325164 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:15.586061 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.585957    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:15.586061 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.586037    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:23.586013509 +0000 UTC m=+18.078657317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:15.685860 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:15.685823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:15.686027 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.685986    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:15.686027 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.686007    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:15.686027 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.686020    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:15.686156 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:15.686074    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:57:23.686057899 +0000 UTC m=+18.178701729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:17.080596 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:17.080563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:17.081042 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:17.080700    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:17.081119 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:17.081068    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:17.081119 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:17.081104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:17.081246 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:17.081184    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:17.081353 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:17.081296    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:19.077491 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:19.077465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:19.077930 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:19.077466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:19.077930 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:19.077577    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:19.077930 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:19.077657    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:19.077930 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:19.077466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:19.077930 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:19.077773    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:21.075349 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:21.075314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:21.075793 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:21.075333    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:21.075793 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:21.075442    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:21.075793 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:21.075334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:21.075793 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:21.075558    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:21.075793 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:21.075604    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:23.077736 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:23.077708    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:23.078178 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:23.077708    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:23.078178 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.077836    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:23.078178 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:23.077709    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:23.078178 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.077901    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:23.078178 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.077974    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:23.648986 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:23.648945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:23.648986 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:23.648996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:23.649224 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.649090    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:23.649224 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.649092    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:23.649224 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.649141    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:39.649125222 +0000 UTC m=+34.141769045 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:23.649224 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.649156    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:57:39.649150126 +0000 UTC m=+34.141793929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:23.749564 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:23.749532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:23.749744 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.749680    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:23.749744 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.749705    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:23.749744 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.749718    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:23.749896 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:23.749770    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:57:39.749756423 +0000 UTC m=+34.242400226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:25.074746 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:25.074710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:25.074746 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:25.074752    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:25.075304 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:25.074710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:25.075304 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:25.074837    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:25.075304 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:25.074895    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:25.075304 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:25.075003    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:27.077960 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:27.077933    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:27.078407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:27.078038    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:27.078407 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:27.078042    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:27.078407 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:27.078052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:27.078407 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:27.078135    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:27.078407 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:27.078205    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:28.313172 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.312867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"49f2cbe9ec70ebfb1e048e0bababcf316fd3e28c7480b401ec9e15b86eed8090"}
Jul 18 01:57:28.313614 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.313281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"5770911cf9b97cb80a1b9d54eb146194b46f97574676ab4f461b4439bc1b0abb"}
Jul 18 01:57:28.314606 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.314579    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nffmj" event={"ID":"b07ceb7e-d2d3-4bf7-8fa4-abb616ffb2f9","Type":"ContainerStarted","Data":"18ef1e95d4fac1ffc683af14449d958661e74400b998c7272b242656c426ed0a"}
Jul 18 01:57:28.316097 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.316078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k87w2" event={"ID":"9e8eb3d7-3e5d-4fef-bc3f-4ee401fcfe05","Type":"ContainerStarted","Data":"fd3c57ad19062cea322c43e0686947f0928e2b7f0b51b9ea5ec228bb195b78f8"}
Jul 18 01:57:28.317561 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.317541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal" event={"ID":"a1303fa5083c2705ceafaaf4e41a3f21","Type":"ContainerStarted","Data":"7dd3d324083c1550112dfc166a88d033449941f5e84fca37b8433ce3c8db3540"}
Jul 18 01:57:28.330960 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.330921    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nffmj" podStartSLOduration=1.7542665899999998 podStartE2EDuration="21.330910603s" totalImagesPullingTime="19.576644013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:28.330533408 +0000 UTC m=+22.823177273" watchObservedRunningTime="2026-07-18 01:57:28.330910603 +0000 UTC m=+22.823554429"
Jul 18 01:57:28.345173 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.345132    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-113.ec2.internal" podStartSLOduration=21.345116356 podStartE2EDuration="21.345116356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:28.344482264 +0000 UTC m=+22.837126093" watchObservedRunningTime="2026-07-18 01:57:28.345116356 +0000 UTC m=+22.837760181"
Jul 18 01:57:28.365489 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:28.365443    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k87w2" podStartSLOduration=2.7315266339999997 podStartE2EDuration="22.365427167s" totalImagesPullingTime="19.633900533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:28.36536704 +0000 UTC m=+22.858010866" watchObservedRunningTime="2026-07-18 01:57:28.365427167 +0000 UTC m=+22.858070994"
Jul 18 01:57:29.075202 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.075052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:29.075202 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.075053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:29.075356 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:29.075180    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:29.075356 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.075229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:29.075356 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:29.075308    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:29.075476 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:29.075399    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:29.322107 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.322025    2576 generic.go:356] "Generic (PLEG): container finished" podID="db62ea2d-a460-48d1-b3ea-16278eff9369" containerID="c2d6d9bcf696fbab928d440a5cf0eb561e3305112441dc9db755fd3eb2bf1582" exitCode=0
Jul 18 01:57:29.322107 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.322066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerDied","Data":"c2d6d9bcf696fbab928d440a5cf0eb561e3305112441dc9db755fd3eb2bf1582"}
Jul 18 01:57:29.324309 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.324284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fd7cq" event={"ID":"294352e3-ce7b-4d1b-a742-1ed861819583","Type":"ContainerStarted","Data":"ecd2453f15390a3d0e9f7d969575c9d5962488310d239fdc48db406d834104a5"}
Jul 18 01:57:29.328072 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.328049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdb4x" event={"ID":"1d780625-cebc-417b-9f06-1f00ec4f972b","Type":"ContainerStarted","Data":"92c591a6c9652b56ad3d4fc4962c9ccd48337fe1f2c6dc2d55b43ccdb9968ab7"}
Jul 18 01:57:29.329909 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.329884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ml5cr" event={"ID":"91dea430-55c6-42e2-9589-148379a893aa","Type":"ContainerStarted","Data":"b4efbfa9c9f3ac563078221742223374421c0b0c8fd71462f5a9bdc8608f8766"}
Jul 18 01:57:29.331896 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.331868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5" event={"ID":"af19b7ac-2be2-4c32-84bc-3b1793db1333","Type":"ContainerStarted","Data":"a1f9b8124e60c0011a4bf4a21d0cf57afd42e5383aec3aa66e1a45807e1989df"}
Jul 18 01:57:29.333577 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.333554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l87cg" event={"ID":"a6887c15-1d53-4eae-91ac-f130359d129d","Type":"ContainerStarted","Data":"46e4f590d882a28ca7928bad3e149299a84fba7caa7399bfe29b5abe6259e181"}
Jul 18 01:57:29.336840 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.336812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"32db4420f2bb60b1c4153b19ef25298cba5490cd4b40a60816ca799d53895d27"}
Jul 18 01:57:29.336938 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.336875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"6ddfa7c1d0971165e176135016c047f5d59f0ffef481489e846f3af189dbe7e3"}
Jul 18 01:57:29.336938 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.336907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"410f4edb5e4a9fb63c7b736d7dbb5a8fa712455a61e05f798820a686750bcd32"}
Jul 18 01:57:29.337034 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.336942    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"1987f974c60b790bebf8c154df21b5229916d9734aa2ff4084aef585d05c6175"}
Jul 18 01:57:29.338299 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.338260    2576 generic.go:356] "Generic (PLEG): container finished" podID="a31f88c21c7864a4b5269fc72fabd5c5" containerID="dd520ce8111e3991fb98e9c93aa8856752b8ec6b2ff6ec1b42c1df6e6b98afec" exitCode=0
Jul 18 01:57:29.338418 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.338300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal" event={"ID":"a31f88c21c7864a4b5269fc72fabd5c5","Type":"ContainerDied","Data":"dd520ce8111e3991fb98e9c93aa8856752b8ec6b2ff6ec1b42c1df6e6b98afec"}
Jul 18 01:57:29.372326 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.372289    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l87cg" podStartSLOduration=3.741816393 podStartE2EDuration="23.372277322s" totalImagesPullingTime="19.630460929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:29.371762283 +0000 UTC m=+23.864406123" watchObservedRunningTime="2026-07-18 01:57:29.372277322 +0000 UTC m=+23.864921168"
Jul 18 01:57:29.405578 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.405541    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ml5cr" podStartSLOduration=2.841949526 podStartE2EDuration="22.405526113s" totalImagesPullingTime="19.563576587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:29.405435314 +0000 UTC m=+23.898079143" watchObservedRunningTime="2026-07-18 01:57:29.405526113 +0000 UTC m=+23.898169943"
Jul 18 01:57:29.405808 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.405790    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fdb4x" podStartSLOduration=3.81047896 podStartE2EDuration="23.405784114s" totalImagesPullingTime="19.595305154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:29.389516426 +0000 UTC m=+23.882160253" watchObservedRunningTime="2026-07-18 01:57:29.405784114 +0000 UTC m=+23.898427940"
Jul 18 01:57:29.440551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.440366    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fd7cq" podStartSLOduration=3.878525662 podStartE2EDuration="23.440345092s" totalImagesPullingTime="19.56181943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:29.439623195 +0000 UTC m=+23.932267020" watchObservedRunningTime="2026-07-18 01:57:29.440345092 +0000 UTC m=+23.932988919"
Jul 18 01:57:29.760377 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:29.760356    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 01:57:30.084023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.083930    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:57:29.760371991Z","UUID":"6922d0f5-c2fe-4f4a-a7d2-3a8b705a359a","Handler":null,"Name":"","Endpoint":""}
Jul 18 01:57:30.085468 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.085446    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 01:57:30.085578 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.085484    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 01:57:30.335453 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.335373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:30.336004 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.335986    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:30.343522 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.343496    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal" event={"ID":"a31f88c21c7864a4b5269fc72fabd5c5","Type":"ContainerStarted","Data":"43578d1542d74d2c5822986ee50fbee1d5c0cd5d5d1a74bf11400dda7972c785"}
Jul 18 01:57:30.345625 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.345593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5" event={"ID":"af19b7ac-2be2-4c32-84bc-3b1793db1333","Type":"ContainerStarted","Data":"ca770451d568d92a7238d1c9f2768da02af2ed1684db0032e76744b84090d325"}
Jul 18 01:57:30.345818 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.345799    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:30.346240 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.346217    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fd7cq"
Jul 18 01:57:30.367842 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:30.367795    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-113.ec2.internal" podStartSLOduration=23.367779947 podStartE2EDuration="23.367779947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:30.366804385 +0000 UTC m=+24.859448224" watchObservedRunningTime="2026-07-18 01:57:30.367779947 +0000 UTC m=+24.860423774"
Jul 18 01:57:31.074991 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:31.074966    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:31.075119 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:31.074967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:31.075119 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:31.075103    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:31.075276 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:31.075138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:31.075276 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:31.075256    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:31.075382 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:31.075346    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:31.349349 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:31.349318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5" event={"ID":"af19b7ac-2be2-4c32-84bc-3b1793db1333","Type":"ContainerStarted","Data":"57f759aa88da500bce18057b2a8765d0cbe88bcc7d48d738e748cc98a159ac2a"}
Jul 18 01:57:31.352448 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:31.352399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"9551949b6b21fd1fcb13d833289d990c6c4b05661d27aa19fd97a915f56e85a6"}
Jul 18 01:57:31.366933 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:31.366885    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97wx5" podStartSLOduration=3.363323895 podStartE2EDuration="25.36686523s" totalImagesPullingTime="22.003541335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:31.36508314 +0000 UTC m=+25.857726968" watchObservedRunningTime="2026-07-18 01:57:31.36686523 +0000 UTC m=+25.859509058"
Jul 18 01:57:33.074720 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:33.074688    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:33.074720 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:33.074716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:33.075319 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:33.074716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:33.075319 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:33.074803    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:33.075319 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:33.074906    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:33.075319 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:33.074987    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:34.362348 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.362080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" event={"ID":"80423d99-c464-4827-883e-a5be2a3cddd7","Type":"ContainerStarted","Data":"ec1cf7547a3d7c63820335c2cdd172c2d7c31c435e50af4683f340bf6dcd97dc"}
Jul 18 01:57:34.362918 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.362398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:34.362918 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.362456    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:34.365884 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.365854    2576 generic.go:356] "Generic (PLEG): container finished" podID="db62ea2d-a460-48d1-b3ea-16278eff9369" containerID="d82b58b1456251c7bccbf96cfd865c37c47d8e1214d96cf957366dc114ca264f" exitCode=0
Jul 18 01:57:34.366023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.365908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerDied","Data":"d82b58b1456251c7bccbf96cfd865c37c47d8e1214d96cf957366dc114ca264f"}
Jul 18 01:57:34.378173 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.378151    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:34.417861 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:34.417820    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds" podStartSLOduration=8.69110164 podStartE2EDuration="28.417806504s" totalImagesPullingTime="19.726704864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:34.417003307 +0000 UTC m=+28.909647134" watchObservedRunningTime="2026-07-18 01:57:34.417806504 +0000 UTC m=+28.910450329"
Jul 18 01:57:35.075425 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.075398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:35.075425 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.075413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:35.075702 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.075419    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:35.075702 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:35.075506    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:35.075702 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:35.075589    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:35.075702 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:35.075648    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:35.368767 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.368694    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:35.382821 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.382797    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:57:35.915091 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.914950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wpn87"]
Jul 18 01:57:35.915248 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.915175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:35.915315 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:35.915278    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:35.918893 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.918870    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tnrz7"]
Jul 18 01:57:35.919002 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.918979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:35.919100 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:35.919078    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:35.919677 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.919654    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wqbsp"]
Jul 18 01:57:35.919778 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.919753    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:35.919865 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:35.919844    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:35.948560 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:35.948543    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ml5cr_91dea430-55c6-42e2-9589-148379a893aa/dns-node-resolver/0.log"
Jul 18 01:57:36.372630 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:36.372560    2576 generic.go:356] "Generic (PLEG): container finished" podID="db62ea2d-a460-48d1-b3ea-16278eff9369" containerID="71f81c8fa64ca45357d94e2195e57a7e852ccd431a1eb01c07530558b22a0c0a" exitCode=0
Jul 18 01:57:36.372989 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:36.372641    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerDied","Data":"71f81c8fa64ca45357d94e2195e57a7e852ccd431a1eb01c07530558b22a0c0a"}
Jul 18 01:57:36.722072 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:36.722051    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdb4x_1d780625-cebc-417b-9f06-1f00ec4f972b/node-ca/0.log"
Jul 18 01:57:37.076782 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:37.076701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:37.076935 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:37.076821    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:37.076935 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:37.076867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:37.077060 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:37.076998    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:37.377037 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:37.377009    2576 generic.go:356] "Generic (PLEG): container finished" podID="db62ea2d-a460-48d1-b3ea-16278eff9369" containerID="183b2f15b8261738ce91dd220cacbe477ea31ff2a666aa13ba108dcf35583f57" exitCode=0
Jul 18 01:57:37.377405 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:37.377094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerDied","Data":"183b2f15b8261738ce91dd220cacbe477ea31ff2a666aa13ba108dcf35583f57"}
Jul 18 01:57:38.074480 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:38.074402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:38.074639 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:38.074526    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:39.074657 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:39.074629    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:39.074657 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:39.074647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:39.075211 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.074756    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:39.075211 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.074868    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:39.672545 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:39.672460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:39.672545 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:39.672511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:39.672759 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.672604    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:39.672759 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.672663    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs podName:678a9215-5982-4c4c-82d2-bc8262c8ab30 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:11.672648658 +0000 UTC m=+66.165292462 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs") pod "network-metrics-daemon-tnrz7" (UID: "678a9215-5982-4c4c-82d2-bc8262c8ab30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:57:39.672759 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.672609    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:39.672759 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.672753    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret podName:b506e3e4-9db5-4d4f-86f4-88cca7974d24 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:11.672734354 +0000 UTC m=+66.165378172 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret") pod "global-pull-secret-syncer-wpn87" (UID: "b506e3e4-9db5-4d4f-86f4-88cca7974d24") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:57:39.773596 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:39.773571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:39.773750 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.773717    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:57:39.773750 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.773734    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:57:39.773750 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.773743    2576 projected.go:196] Error preparing data for projected volume kube-api-access-2c9fr for pod openshift-network-diagnostics/network-check-target-wqbsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:39.773885 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:39.773793    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr podName:2d6e36ac-7a25-4c07-a539-e4d8e78a405a nodeName:}" failed. No retries permitted until 2026-07-18 01:58:11.773779547 +0000 UTC m=+66.266423360 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2c9fr" (UniqueName: "kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr") pod "network-check-target-wqbsp" (UID: "2d6e36ac-7a25-4c07-a539-e4d8e78a405a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:57:40.074882 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:40.074853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:40.075299 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:40.074999    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:41.074921 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:41.074888    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:41.075403 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:41.074902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:41.075403 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:41.075045    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:41.075403 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:41.075100    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:42.075306 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:42.075274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:42.075745 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:42.075380    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:42.395929 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:42.395900    2576 generic.go:356] "Generic (PLEG): container finished" podID="db62ea2d-a460-48d1-b3ea-16278eff9369" containerID="d66e51afa3a3fd19049c814bf54e7adf5c23a5eedfe20c2847919944e65f7d33" exitCode=0
Jul 18 01:57:42.396013 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:42.395940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerDied","Data":"d66e51afa3a3fd19049c814bf54e7adf5c23a5eedfe20c2847919944e65f7d33"}
Jul 18 01:57:43.077799 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:43.077772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:43.077799 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:43.077789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:43.078179 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:43.077866    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:43.078179 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:43.077994    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:43.401469 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:43.401252    2576 generic.go:356] "Generic (PLEG): container finished" podID="db62ea2d-a460-48d1-b3ea-16278eff9369" containerID="cdc8bb3c1da401a61a56e1717746f033c4494f621a6f4c07aa11bd593103d915" exitCode=0
Jul 18 01:57:43.401469 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:43.401333    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerDied","Data":"cdc8bb3c1da401a61a56e1717746f033c4494f621a6f4c07aa11bd593103d915"}
Jul 18 01:57:44.075151 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:44.075119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:44.075341 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:44.075242    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:44.406906 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:44.406835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" event={"ID":"db62ea2d-a460-48d1-b3ea-16278eff9369","Type":"ContainerStarted","Data":"0ddf7254885672238aaf508892a94bab1a4f03461688df0ad3054644b62c1bd0"}
Jul 18 01:57:44.429912 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:44.429866    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d8rc6" podStartSLOduration=6.68983117 podStartE2EDuration="37.429851347s" totalImagesPullingTime="30.740020177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:44.429180966 +0000 UTC m=+38.921824791" watchObservedRunningTime="2026-07-18 01:57:44.429851347 +0000 UTC m=+38.922495173"
Jul 18 01:57:45.078055 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:45.078020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:45.078257 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:45.078135    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:45.078257 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:45.078182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:45.078388 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:45.078279    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:46.074986 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:46.074960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:46.075409 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:46.075048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:47.079844 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:47.079812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:47.079844 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:47.079838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:47.080388 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:47.079911    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:47.080388 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:47.080057    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:48.075396 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:48.075366    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:48.075567 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:48.075455    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:49.077554 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:49.077523    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:49.077959 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:49.077620    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:49.077959 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:49.077692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:49.077959 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:49.077775    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:50.074755 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:50.074724    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:50.074899 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:50.074818    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:51.075034 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:51.075001    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:51.075506 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:51.075143    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:51.075506 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:51.075178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:51.075506 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:51.075288    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:52.074495 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:52.074465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:52.074652 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:52.074558    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:53.075210 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:53.075165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:53.075592 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:53.075176    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:53.075592 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:53.075321    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:53.075592 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:53.075358    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:54.075265 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:54.075237    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:54.075637 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:54.075332    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:55.075141 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:55.075105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:55.075141 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:55.075126    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:55.075595 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:55.075275    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:55.075595 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:55.075379    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:56.075042 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:56.075009    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:56.075233 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:56.075101    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:57.076065 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:57.076028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:57.076548 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:57.076108    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:57.076548 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:57.076229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:57.076548 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:57.076338    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:57:58.075390 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:58.075358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:57:58.075584 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:58.075472    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:57:59.075033 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:59.074997    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:57:59.075410 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:59.075106    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:57:59.075410 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:57:59.075136    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:57:59.075410 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:57:59.075217    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:58:00.074848 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:00.074817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:00.075084 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:00.074929    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wqbsp" podUID="2d6e36ac-7a25-4c07-a539-e4d8e78a405a"
Jul 18 01:58:01.074747 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.074717    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:58:01.074747 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.074737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:58:01.074938 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:01.074813    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpn87" podUID="b506e3e4-9db5-4d4f-86f4-88cca7974d24"
Jul 18 01:58:01.074977 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:01.074951    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrz7" podUID="678a9215-5982-4c4c-82d2-bc8262c8ab30"
Jul 18 01:58:01.744138 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.744082    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-113.ec2.internal" event="NodeReady"
Jul 18 01:58:01.744559 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.744180    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 01:58:01.792692 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.792666    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6r2lr"]
Jul 18 01:58:01.795325 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.795307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.797955 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.797936    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 01:58:01.798187 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.798172    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-96mg4\""
Jul 18 01:58:01.798348 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.798330    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 01:58:01.798628 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.798611    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nktd6"]
Jul 18 01:58:01.802143 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.800823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.804040 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.803734    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rd42x\""
Jul 18 01:58:01.804162 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.804139    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 01:58:01.804366 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.804338    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 01:58:01.804457 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.804403    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 01:58:01.804639 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.804622    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 01:58:01.809589 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.809570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6r2lr"]
Jul 18 01:58:01.812829 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.812806    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nktd6"]
Jul 18 01:58:01.838649 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838630    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-data-volume\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.838718 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838661    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hshbw\" (UniqueName: \"kubernetes.io/projected/181cb719-070d-4ea8-800c-c7b54a6a915b-kube-api-access-hshbw\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.838718 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.838718 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181cb719-070d-4ea8-800c-c7b54a6a915b-tmp-dir\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.838833 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/181cb719-070d-4ea8-800c-c7b54a6a915b-config-volume\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.838867 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-crio-socket\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.838899 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/181cb719-070d-4ea8-800c-c7b54a6a915b-metrics-tls\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.838899 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.838957 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.838914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kl5k\" (UniqueName: \"kubernetes.io/projected/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-kube-api-access-4kl5k\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.895677 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.895657    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mtpxd"]
Jul 18 01:58:01.897298 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.897284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:01.899616 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.899596    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 01:58:01.899928 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.899908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 01:58:01.900018 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.899960    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 01:58:01.900018 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.900002    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcv7c\""
Jul 18 01:58:01.907937 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.907918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mtpxd"]
Jul 18 01:58:01.939429 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.939525 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc9fb34-b2f3-467a-ae70-77afade7a241-cert\") pod \"ingress-canary-mtpxd\" (UID: \"dfc9fb34-b2f3-467a-ae70-77afade7a241\") " pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:01.939525 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181cb719-070d-4ea8-800c-c7b54a6a915b-tmp-dir\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.939525 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/181cb719-070d-4ea8-800c-c7b54a6a915b-config-volume\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.939689 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-crio-socket\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.939689 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glb2r\" (UniqueName: \"kubernetes.io/projected/dfc9fb34-b2f3-467a-ae70-77afade7a241-kube-api-access-glb2r\") pod \"ingress-canary-mtpxd\" (UID: \"dfc9fb34-b2f3-467a-ae70-77afade7a241\") " pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:01.939689 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/181cb719-070d-4ea8-800c-c7b54a6a915b-metrics-tls\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.939689 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.939852 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/181cb719-070d-4ea8-800c-c7b54a6a915b-tmp-dir\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.939852 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-crio-socket\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.939852 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kl5k\" (UniqueName: \"kubernetes.io/projected/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-kube-api-access-4kl5k\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.939973 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-data-volume\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.939973 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.939895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hshbw\" (UniqueName: \"kubernetes.io/projected/181cb719-070d-4ea8-800c-c7b54a6a915b-kube-api-access-hshbw\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.940069 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.940055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/181cb719-070d-4ea8-800c-c7b54a6a915b-config-volume\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.940216 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.940174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-data-volume\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.940308 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.940181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.943256 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.943240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.943343 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.943238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/181cb719-070d-4ea8-800c-c7b54a6a915b-metrics-tls\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:01.949474 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.949456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kl5k\" (UniqueName: \"kubernetes.io/projected/7f0d691e-e9f8-484e-ad9b-00acdd3bf10c-kube-api-access-4kl5k\") pod \"insights-runtime-extractor-nktd6\" (UID: \"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c\") " pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:01.949665 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:01.949646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hshbw\" (UniqueName: \"kubernetes.io/projected/181cb719-070d-4ea8-800c-c7b54a6a915b-kube-api-access-hshbw\") pod \"dns-default-6r2lr\" (UID: \"181cb719-070d-4ea8-800c-c7b54a6a915b\") " pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:02.040777 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.040730    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc9fb34-b2f3-467a-ae70-77afade7a241-cert\") pod \"ingress-canary-mtpxd\" (UID: \"dfc9fb34-b2f3-467a-ae70-77afade7a241\") " pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:02.040855 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.040789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glb2r\" (UniqueName: \"kubernetes.io/projected/dfc9fb34-b2f3-467a-ae70-77afade7a241-kube-api-access-glb2r\") pod \"ingress-canary-mtpxd\" (UID: \"dfc9fb34-b2f3-467a-ae70-77afade7a241\") " pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:02.042622 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.042604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc9fb34-b2f3-467a-ae70-77afade7a241-cert\") pod \"ingress-canary-mtpxd\" (UID: \"dfc9fb34-b2f3-467a-ae70-77afade7a241\") " pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:02.048221 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.048179    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glb2r\" (UniqueName: \"kubernetes.io/projected/dfc9fb34-b2f3-467a-ae70-77afade7a241-kube-api-access-glb2r\") pod \"ingress-canary-mtpxd\" (UID: \"dfc9fb34-b2f3-467a-ae70-77afade7a241\") " pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:02.074367 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.074346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:02.077086 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.077070    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:58:02.077183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.077105    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:58:02.077183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.077130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr2dl\""
Jul 18 01:58:02.107538 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.107508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:02.112861 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.112847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nktd6"
Jul 18 01:58:02.205885 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.205616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mtpxd"
Jul 18 01:58:02.272044 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.271989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nktd6"]
Jul 18 01:58:02.283428 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.283404    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6r2lr"]
Jul 18 01:58:02.286709 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:02.286686    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181cb719_070d_4ea8_800c_c7b54a6a915b.slice/crio-fde7cafbc69a3f59e11f0ef26dec23c045344948556ec41004ec527a6b1bc2fb WatchSource:0}: Error finding container fde7cafbc69a3f59e11f0ef26dec23c045344948556ec41004ec527a6b1bc2fb: Status 404 returned error can't find the container with id fde7cafbc69a3f59e11f0ef26dec23c045344948556ec41004ec527a6b1bc2fb
Jul 18 01:58:02.332917 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.332888    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mtpxd"]
Jul 18 01:58:02.335796 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:02.335772    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc9fb34_b2f3_467a_ae70_77afade7a241.slice/crio-5fe1ba45b55fe35ad398369a3727fb2dbe95a6642bb83cb2b09a23b61ed92ae2 WatchSource:0}: Error finding container 5fe1ba45b55fe35ad398369a3727fb2dbe95a6642bb83cb2b09a23b61ed92ae2: Status 404 returned error can't find the container with id 5fe1ba45b55fe35ad398369a3727fb2dbe95a6642bb83cb2b09a23b61ed92ae2
Jul 18 01:58:02.446698 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.446664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nktd6" event={"ID":"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c","Type":"ContainerStarted","Data":"eab27dd7ae9eb35823275606c5579738c53908926fd632dfca2d0b421eed878a"}
Jul 18 01:58:02.447759 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.447736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mtpxd" event={"ID":"dfc9fb34-b2f3-467a-ae70-77afade7a241","Type":"ContainerStarted","Data":"5fe1ba45b55fe35ad398369a3727fb2dbe95a6642bb83cb2b09a23b61ed92ae2"}
Jul 18 01:58:02.448789 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:02.448770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6r2lr" event={"ID":"181cb719-070d-4ea8-800c-c7b54a6a915b","Type":"ContainerStarted","Data":"fde7cafbc69a3f59e11f0ef26dec23c045344948556ec41004ec527a6b1bc2fb"}
Jul 18 01:58:03.077047 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:03.077011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:58:03.077518 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:03.077321    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:58:03.081102 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:03.081074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:58:03.081348 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:03.081331    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:58:03.081543 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:03.081527    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5xbv\""
Jul 18 01:58:04.970701 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:04.970672    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"]
Jul 18 01:58:04.972784 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:04.972767    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:04.975408 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:04.975383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 18 01:58:04.975521 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:04.975418    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cz4nr\""
Jul 18 01:58:04.982037 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:04.981989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"]
Jul 18 01:58:05.075208 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:05.075163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-7tvqs\" (UID: \"15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:05.176446 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:05.176403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-7tvqs\" (UID: \"15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:05.179351 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:05.179323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-7tvqs\" (UID: \"15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:05.282390 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:05.282319    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:07.397365 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:07.397340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2s9ds"
Jul 18 01:58:08.568958 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:08.568503    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"]
Jul 18 01:58:08.571219 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:08.571135    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b3b6fe_a4ac_4887_a8b1_7e5c6f2b1022.slice/crio-072c117067ed8b80a3cc2320cccefb23b6dd62928113da5c81041b2440829945 WatchSource:0}: Error finding container 072c117067ed8b80a3cc2320cccefb23b6dd62928113da5c81041b2440829945: Status 404 returned error can't find the container with id 072c117067ed8b80a3cc2320cccefb23b6dd62928113da5c81041b2440829945
Jul 18 01:58:09.197588 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.197555    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75d568c9dd-zhrss"]
Jul 18 01:58:09.199612 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.199588    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.206212 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.204548    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 18 01:58:09.206212 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.204780    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 18 01:58:09.208220 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.207874    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 18 01:58:09.208220 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.208121    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 18 01:58:09.212215 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.210929    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 18 01:58:09.212215 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.211327    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 18 01:58:09.212215 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.211939    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 18 01:58:09.220137 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.212526    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-22tsw\""
Jul 18 01:58:09.220137 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.217996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75d568c9dd-zhrss"]
Jul 18 01:58:09.230860 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.230838    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 18 01:58:09.306442 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-service-ca\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.306576 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-config\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.306576 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-trusted-ca-bundle\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.306576 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-serving-cert\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.306741 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-oauth-config\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.306741 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-oauth-serving-cert\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.306741 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.306708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzmn\" (UniqueName: \"kubernetes.io/projected/ee320774-08ef-47a5-bf97-53a80ebd01e2-kube-api-access-vkzmn\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407172 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-config\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407172 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-trusted-ca-bundle\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407416 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-serving-cert\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407416 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407356    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-oauth-config\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407416 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-oauth-serving-cert\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407566 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzmn\" (UniqueName: \"kubernetes.io/projected/ee320774-08ef-47a5-bf97-53a80ebd01e2-kube-api-access-vkzmn\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.407566 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-service-ca\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.408505 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.407931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-config\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.408505 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.408040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-trusted-ca-bundle\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.408852 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.408821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-oauth-serving-cert\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.408971 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.408949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-service-ca\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.410785 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.410761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-serving-cert\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.412276 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.412250    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-oauth-config\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.417671 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.417647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzmn\" (UniqueName: \"kubernetes.io/projected/ee320774-08ef-47a5-bf97-53a80ebd01e2-kube-api-access-vkzmn\") pod \"console-75d568c9dd-zhrss\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") " pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.469239 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.469151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nktd6" event={"ID":"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c","Type":"ContainerStarted","Data":"5e1562927b28c13a475513dfec0265ed3543aa331d189c211187975206f46bf2"}
Jul 18 01:58:09.471329 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.471300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs" event={"ID":"15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022","Type":"ContainerStarted","Data":"072c117067ed8b80a3cc2320cccefb23b6dd62928113da5c81041b2440829945"}
Jul 18 01:58:09.473934 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.473857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mtpxd" event={"ID":"dfc9fb34-b2f3-467a-ae70-77afade7a241","Type":"ContainerStarted","Data":"322e4cf7730564abd70f198a159e5264d1af2b83bc2bb8495cba7f37bf56c352"}
Jul 18 01:58:09.476986 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.476383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6r2lr" event={"ID":"181cb719-070d-4ea8-800c-c7b54a6a915b","Type":"ContainerStarted","Data":"c3ac3e4b7b4d535e411d898bfbf677c0fff39d43544546d8133c4ac47f22241e"}
Jul 18 01:58:09.476986 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.476435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6r2lr" event={"ID":"181cb719-070d-4ea8-800c-c7b54a6a915b","Type":"ContainerStarted","Data":"6e318f572b18914f8ecbd706bc5623aba81a7531aacb352ad2e9a60d2021fb71"}
Jul 18 01:58:09.476986 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.476932    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:09.489628 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.489581    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mtpxd" podStartSLOduration=2.387120521 podStartE2EDuration="8.489566339s" totalImagesPullingTime="6.102445818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:09.489223479 +0000 UTC m=+63.981867309" watchObservedRunningTime="2026-07-18 01:58:09.489566339 +0000 UTC m=+63.982210166"
Jul 18 01:58:09.506652 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.506034    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6r2lr" podStartSLOduration=2.354436523 podStartE2EDuration="8.50601566s" totalImagesPullingTime="6.151579137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:09.505054848 +0000 UTC m=+63.997698675" watchObservedRunningTime="2026-07-18 01:58:09.50601566 +0000 UTC m=+63.998659488"
Jul 18 01:58:09.529794 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.529570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:09.826901 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:09.826866    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75d568c9dd-zhrss"]
Jul 18 01:58:09.831988 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:09.831953    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee320774_08ef_47a5_bf97_53a80ebd01e2.slice/crio-5b18861c8dcda4fa13fcba8c6f143a63af85be86ed4d5d4358fb7fa0d914e48d WatchSource:0}: Error finding container 5b18861c8dcda4fa13fcba8c6f143a63af85be86ed4d5d4358fb7fa0d914e48d: Status 404 returned error can't find the container with id 5b18861c8dcda4fa13fcba8c6f143a63af85be86ed4d5d4358fb7fa0d914e48d
Jul 18 01:58:10.480533 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:10.480491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs" event={"ID":"15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022","Type":"ContainerStarted","Data":"42a1722f6652717d3868307de05717604038ab9ccf761eadf184443255730118"}
Jul 18 01:58:10.480817 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:10.480770    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:10.481848 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:10.481800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d568c9dd-zhrss" event={"ID":"ee320774-08ef-47a5-bf97-53a80ebd01e2","Type":"ContainerStarted","Data":"5b18861c8dcda4fa13fcba8c6f143a63af85be86ed4d5d4358fb7fa0d914e48d"}
Jul 18 01:58:10.485601 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:10.485579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs"
Jul 18 01:58:10.495988 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:10.495944    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-7tvqs" podStartSLOduration=5.350935734 podStartE2EDuration="6.495926263s" totalImagesPullingTime="1.144990529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:10.495666364 +0000 UTC m=+64.988310191" watchObservedRunningTime="2026-07-18 01:58:10.495926263 +0000 UTC m=+64.988570089"
Jul 18 01:58:11.035490 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.034982    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"]
Jul 18 01:58:11.048849 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.048816    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"]
Jul 18 01:58:11.049020 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.048944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.051847 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.051822    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 18 01:58:11.051968 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.051914    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 01:58:11.052035 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.051822    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 01:58:11.052238 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.052218    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 01:58:11.053096 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.052947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 18 01:58:11.053096 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.053042    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-msjn2\""
Jul 18 01:58:11.121480 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.121447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x88mw\" (UniqueName: \"kubernetes.io/projected/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-kube-api-access-x88mw\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.121626 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.121510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.121675 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.121639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.121785 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.121673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.222566 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.222532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.222721 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.222640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.222721 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.222679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.222838 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.222731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x88mw\" (UniqueName: \"kubernetes.io/projected/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-kube-api-access-x88mw\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.223495 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.223444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.225533 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.225493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.225836 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.225817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.232114 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.232086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x88mw\" (UniqueName: \"kubernetes.io/projected/05f9043f-2f88-4a2b-a45c-70eb7dc5136b-kube-api-access-x88mw\") pod \"prometheus-operator-5945b5cfd-w44mp\" (UID: \"05f9043f-2f88-4a2b-a45c-70eb7dc5136b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.359912 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.359841    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"
Jul 18 01:58:11.728081 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.728047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:58:11.728247 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.728096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:58:11.731206 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.731165    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:58:11.732244 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.732222    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:58:11.741148 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.741125    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b506e3e4-9db5-4d4f-86f4-88cca7974d24-original-pull-secret\") pod \"global-pull-secret-syncer-wpn87\" (UID: \"b506e3e4-9db5-4d4f-86f4-88cca7974d24\") " pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:58:11.741239 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.741138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678a9215-5982-4c4c-82d2-bc8262c8ab30-metrics-certs\") pod \"network-metrics-daemon-tnrz7\" (UID: \"678a9215-5982-4c4c-82d2-bc8262c8ab30\") " pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:58:11.793275 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.793251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpn87"
Jul 18 01:58:11.802969 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.802938    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5xbv\""
Jul 18 01:58:11.811321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.811305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrz7"
Jul 18 01:58:11.828635 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.828614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:11.831167 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.831149    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:58:11.841224 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.841207    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:58:11.851667 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.851639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c9fr\" (UniqueName: \"kubernetes.io/projected/2d6e36ac-7a25-4c07-a539-e4d8e78a405a-kube-api-access-2c9fr\") pod \"network-check-target-wqbsp\" (UID: \"2d6e36ac-7a25-4c07-a539-e4d8e78a405a\") " pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:11.985224 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.985187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr2dl\""
Jul 18 01:58:11.993595 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:11.993566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:12.142822 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.142684    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tnrz7"]
Jul 18 01:58:12.148240 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:12.148184    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678a9215_5982_4c4c_82d2_bc8262c8ab30.slice/crio-231272786bfe4e4394a873d6ed22b3fc9b29e2dc77c581c747549256c761c21e WatchSource:0}: Error finding container 231272786bfe4e4394a873d6ed22b3fc9b29e2dc77c581c747549256c761c21e: Status 404 returned error can't find the container with id 231272786bfe4e4394a873d6ed22b3fc9b29e2dc77c581c747549256c761c21e
Jul 18 01:58:12.169714 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.169532    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-w44mp"]
Jul 18 01:58:12.173314 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:12.173275    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f9043f_2f88_4a2b_a45c_70eb7dc5136b.slice/crio-9fce5fcb217e53b53decd7efc39e2039fa1cb59a0087a01b1924c282b58cbed9 WatchSource:0}: Error finding container 9fce5fcb217e53b53decd7efc39e2039fa1cb59a0087a01b1924c282b58cbed9: Status 404 returned error can't find the container with id 9fce5fcb217e53b53decd7efc39e2039fa1cb59a0087a01b1924c282b58cbed9
Jul 18 01:58:12.184070 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.184008    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wpn87"]
Jul 18 01:58:12.187542 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:12.187514    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb506e3e4_9db5_4d4f_86f4_88cca7974d24.slice/crio-3794489b37202034b2267f2a4a378dc128091570f8024fa7a1c6294c173af5da WatchSource:0}: Error finding container 3794489b37202034b2267f2a4a378dc128091570f8024fa7a1c6294c173af5da: Status 404 returned error can't find the container with id 3794489b37202034b2267f2a4a378dc128091570f8024fa7a1c6294c173af5da
Jul 18 01:58:12.213337 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.213313    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wqbsp"]
Jul 18 01:58:12.216957 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:12.216923    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6e36ac_7a25_4c07_a539_e4d8e78a405a.slice/crio-76fd90df97af067735fc31a1849a9ba09c6f2ba0d0c47d33dd370f20aec31cae WatchSource:0}: Error finding container 76fd90df97af067735fc31a1849a9ba09c6f2ba0d0c47d33dd370f20aec31cae: Status 404 returned error can't find the container with id 76fd90df97af067735fc31a1849a9ba09c6f2ba0d0c47d33dd370f20aec31cae
Jul 18 01:58:12.489807 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.489772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp" event={"ID":"05f9043f-2f88-4a2b-a45c-70eb7dc5136b","Type":"ContainerStarted","Data":"9fce5fcb217e53b53decd7efc39e2039fa1cb59a0087a01b1924c282b58cbed9"}
Jul 18 01:58:12.491270 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.491237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wqbsp" event={"ID":"2d6e36ac-7a25-4c07-a539-e4d8e78a405a","Type":"ContainerStarted","Data":"76fd90df97af067735fc31a1849a9ba09c6f2ba0d0c47d33dd370f20aec31cae"}
Jul 18 01:58:12.492722 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.492695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wpn87" event={"ID":"b506e3e4-9db5-4d4f-86f4-88cca7974d24","Type":"ContainerStarted","Data":"3794489b37202034b2267f2a4a378dc128091570f8024fa7a1c6294c173af5da"}
Jul 18 01:58:12.494875 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.494849    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nktd6" event={"ID":"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c","Type":"ContainerStarted","Data":"70d150960d96e84b47e11235cbfc1157593c76ef34ccf4fa06b72199868964c8"}
Jul 18 01:58:12.496247 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:12.496223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnrz7" event={"ID":"678a9215-5982-4c4c-82d2-bc8262c8ab30","Type":"ContainerStarted","Data":"231272786bfe4e4394a873d6ed22b3fc9b29e2dc77c581c747549256c761c21e"}
Jul 18 01:58:20.486323 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.486289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6r2lr"
Jul 18 01:58:20.529560 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.529526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nktd6" event={"ID":"7f0d691e-e9f8-484e-ad9b-00acdd3bf10c","Type":"ContainerStarted","Data":"fc152212372c681400131929d6bd1e63b46eaf04e0f50d54baca19b3bbca1016"}
Jul 18 01:58:20.531575 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.531538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnrz7" event={"ID":"678a9215-5982-4c4c-82d2-bc8262c8ab30","Type":"ContainerStarted","Data":"11731892cbbaaaf40d5408223d6dfee8c1ea70c12f4bc86761e7b3e417a22e00"}
Jul 18 01:58:20.531712 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.531587    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnrz7" event={"ID":"678a9215-5982-4c4c-82d2-bc8262c8ab30","Type":"ContainerStarted","Data":"dc5a7c80218d2685c832137790a87ffca6d90fdba7efeb86a2bb4a41e873b97f"}
Jul 18 01:58:20.533382 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.533358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d568c9dd-zhrss" event={"ID":"ee320774-08ef-47a5-bf97-53a80ebd01e2","Type":"ContainerStarted","Data":"aa06f5d33cc46a2e5a6fa2e67ce65545ca29580827fd26bfc4b212b508a4db17"}
Jul 18 01:58:20.535447 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.535419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp" event={"ID":"05f9043f-2f88-4a2b-a45c-70eb7dc5136b","Type":"ContainerStarted","Data":"dbc83493e765651dae06d0ec5c21a44321ba8720f7bc41e541fcfeaed6c9acae"}
Jul 18 01:58:20.535590 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.535465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp" event={"ID":"05f9043f-2f88-4a2b-a45c-70eb7dc5136b","Type":"ContainerStarted","Data":"5deb75aa59fdebd0f0f3fac65d795a28772aa9ce0f45be016dfdfd7b479030fe"}
Jul 18 01:58:20.537019 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.536992    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wqbsp" event={"ID":"2d6e36ac-7a25-4c07-a539-e4d8e78a405a","Type":"ContainerStarted","Data":"3a10e048b88ad1085807913a52fbd74b0881d8f1827db32e97b2f70ca8bff621"}
Jul 18 01:58:20.537158 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.537067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:20.538437 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.538417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wpn87" event={"ID":"b506e3e4-9db5-4d4f-86f4-88cca7974d24","Type":"ContainerStarted","Data":"aa84191441befbb9a5a9faeb994a8134406d112861d7e34602fdc98248565d48"}
Jul 18 01:58:20.549942 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.549877    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nktd6" podStartSLOduration=2.240669461 podStartE2EDuration="19.549861158s" totalImagesPullingTime="17.309191697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:58:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:20.54910135 +0000 UTC m=+75.041745178" watchObservedRunningTime="2026-07-18 01:58:20.549861158 +0000 UTC m=+75.042504976"
Jul 18 01:58:20.564702 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.564665    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wqbsp" podStartSLOduration=67.031820712 podStartE2EDuration="1m14.56465599s" totalImagesPullingTime="7.532835278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:20.564051791 +0000 UTC m=+75.056695621" watchObservedRunningTime="2026-07-18 01:58:20.56465599 +0000 UTC m=+75.057299815"
Jul 18 01:58:20.580280 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.580245    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-w44mp" podStartSLOduration=2.014722137 podStartE2EDuration="9.580233822s" totalImagesPullingTime="7.565511685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:20.579208813 +0000 UTC m=+75.071852630" watchObservedRunningTime="2026-07-18 01:58:20.580233822 +0000 UTC m=+75.072877648"
Jul 18 01:58:20.593801 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.593768    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wpn87" podStartSLOduration=66.023899456 podStartE2EDuration="1m13.593757504s" totalImagesPullingTime="7.569858048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:20.592958627 +0000 UTC m=+75.085602454" watchObservedRunningTime="2026-07-18 01:58:20.593757504 +0000 UTC m=+75.086401329"
Jul 18 01:58:20.611590 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.611552    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75d568c9dd-zhrss" podStartSLOduration=1.702000211 podStartE2EDuration="11.611537775s" totalImagesPullingTime="9.909537564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:20.610689287 +0000 UTC m=+75.103333116" watchObservedRunningTime="2026-07-18 01:58:20.611537775 +0000 UTC m=+75.104181601"
Jul 18 01:58:20.625436 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:20.625405    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tnrz7" podStartSLOduration=71.082992044 podStartE2EDuration="1m14.625393744s" totalImagesPullingTime="3.5424017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:20.624646562 +0000 UTC m=+75.117290390" watchObservedRunningTime="2026-07-18 01:58:20.625393744 +0000 UTC m=+75.118037570"
Jul 18 01:58:22.378426 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.378392    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"]
Jul 18 01:58:22.381598 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.381582    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.384128 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.384099    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 18 01:58:22.384128 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.384119    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 18 01:58:22.384312 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.384295    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4zt7n\""
Jul 18 01:58:22.398955 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.398935    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"]
Jul 18 01:58:22.425692 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.425670    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-55z2j"]
Jul 18 01:58:22.428892 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.428877    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"]
Jul 18 01:58:22.429018 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.429004    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.431490 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.431469    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 01:58:22.431570 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.431497    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 01:58:22.431570 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.431548    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 01:58:22.431570 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.431559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s9bfv\""
Jul 18 01:58:22.431807 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.431792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.433911 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.433892    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 18 01:58:22.434445 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.434425    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 18 01:58:22.434525 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.434472    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8dbq7\""
Jul 18 01:58:22.434700 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.434686    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 18 01:58:22.441057 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.441041    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"]
Jul 18 01:58:22.510200 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-root\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510295 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.510339 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/350476df-fab6-46ec-ba57-27873edae6d7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.510339 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.510405 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/350476df-fab6-46ec-ba57-27873edae6d7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.510438 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b95dq\" (UniqueName: \"kubernetes.io/projected/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-api-access-b95dq\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.510469 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350476df-fab6-46ec-ba57-27873edae6d7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.510504 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grhp\" (UniqueName: \"kubernetes.io/projected/350476df-fab6-46ec-ba57-27873edae6d7-kube-api-access-4grhp\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.510551 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.510583 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510583 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510640 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-accelerators-collector-config\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510640 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.510640 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-wtmp\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510732 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvprc\" (UniqueName: \"kubernetes.io/projected/0fe0498d-6e61-41c2-add3-ee8a1db91da4-kube-api-access-cvprc\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510732 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.510732 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-sys\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510732 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fe0498d-6e61-41c2-add3-ee8a1db91da4-metrics-client-ca\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.510732 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.510714    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-textfile\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.610967 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.610946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-accelerators-collector-config\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611082 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.610974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611082 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.610993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-wtmp\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611082 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvprc\" (UniqueName: \"kubernetes.io/projected/0fe0498d-6e61-41c2-add3-ee8a1db91da4-kube-api-access-cvprc\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611082 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-wtmp\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-sys\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-sys\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fe0498d-6e61-41c2-add3-ee8a1db91da4-metrics-client-ca\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-textfile\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611321 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-root\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611602 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611602 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/350476df-fab6-46ec-ba57-27873edae6d7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.611602 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611602 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611407    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0fe0498d-6e61-41c2-add3-ee8a1db91da4-root\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611602 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/350476df-fab6-46ec-ba57-27873edae6d7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.611602 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611412    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611623    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-accelerators-collector-config\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-textfile\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b95dq\" (UniqueName: \"kubernetes.io/projected/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-api-access-b95dq\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fe0498d-6e61-41c2-add3-ee8a1db91da4-metrics-client-ca\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350476df-fab6-46ec-ba57-27873edae6d7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grhp\" (UniqueName: \"kubernetes.io/projected/350476df-fab6-46ec-ba57-27873edae6d7-kube-api-access-4grhp\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.611897 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.611839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.612409 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.612374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.612509 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:22.612487    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 01:58:22.612579 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.612555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350476df-fab6-46ec-ba57-27873edae6d7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.612635 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.612543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.612635 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:22.612562    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls podName:0fe0498d-6e61-41c2-add3-ee8a1db91da4 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:23.112542922 +0000 UTC m=+77.605186726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls") pod "node-exporter-55z2j" (UID: "0fe0498d-6e61-41c2-add3-ee8a1db91da4") : secret "node-exporter-tls" not found
Jul 18 01:58:22.614076 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.614053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.614216 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.614174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/350476df-fab6-46ec-ba57-27873edae6d7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.614301 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.614284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/350476df-fab6-46ec-ba57-27873edae6d7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.614552 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.614531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.614990 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.614970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.623833 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.623812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvprc\" (UniqueName: \"kubernetes.io/projected/0fe0498d-6e61-41c2-add3-ee8a1db91da4-kube-api-access-cvprc\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:22.624172 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.624151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b95dq\" (UniqueName: \"kubernetes.io/projected/acda8baf-5ef2-46bb-bb7a-90bed39b16ff-kube-api-access-b95dq\") pod \"kube-state-metrics-56c5594674-4pvj8\" (UID: \"acda8baf-5ef2-46bb-bb7a-90bed39b16ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.624716 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.624693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grhp\" (UniqueName: \"kubernetes.io/projected/350476df-fab6-46ec-ba57-27873edae6d7-kube-api-access-4grhp\") pod \"openshift-state-metrics-79f6958cdf-5t5rk\" (UID: \"350476df-fab6-46ec-ba57-27873edae6d7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.691010 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.690952    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"
Jul 18 01:58:22.743671 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.743647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"
Jul 18 01:58:22.825778 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.825754    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk"]
Jul 18 01:58:22.829329 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:22.829304    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350476df_fab6_46ec_ba57_27873edae6d7.slice/crio-493b816277a0cd42678967683a0bf30d255d528fad3a7869f578541b0d09737d WatchSource:0}: Error finding container 493b816277a0cd42678967683a0bf30d255d528fad3a7869f578541b0d09737d: Status 404 returned error can't find the container with id 493b816277a0cd42678967683a0bf30d255d528fad3a7869f578541b0d09737d
Jul 18 01:58:22.873400 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:22.873380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-4pvj8"]
Jul 18 01:58:22.875501 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:22.875480    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacda8baf_5ef2_46bb_bb7a_90bed39b16ff.slice/crio-9b417cf493824dce827e70b4782c48f2f4fbb52421089443c0c5629dcce1e267 WatchSource:0}: Error finding container 9b417cf493824dce827e70b4782c48f2f4fbb52421089443c0c5629dcce1e267: Status 404 returned error can't find the container with id 9b417cf493824dce827e70b4782c48f2f4fbb52421089443c0c5629dcce1e267
Jul 18 01:58:23.115900 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.115868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:23.116031 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:23.115990    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 18 01:58:23.116072 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:23.116041    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls podName:0fe0498d-6e61-41c2-add3-ee8a1db91da4 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:24.11602731 +0000 UTC m=+78.608671115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls") pod "node-exporter-55z2j" (UID: "0fe0498d-6e61-41c2-add3-ee8a1db91da4") : secret "node-exporter-tls" not found
Jul 18 01:58:23.442585 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.442533    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:58:23.446657 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.446461    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.451049 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.451008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 18 01:58:23.451313 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.451295    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.451656    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.451749    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-plsfz\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.451845    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.451934    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.452084    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.452244    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.452379    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 18 01:58:23.453637 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.452526    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 18 01:58:23.461401 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.461307    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:58:23.518877 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.518846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvhxh\" (UniqueName: \"kubernetes.io/projected/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-kube-api-access-fvhxh\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.518895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.518930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.518952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.518987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-config-volume\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519307 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519307 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-config-out\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519307 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519307 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519307 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-web-config\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519307 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.519591 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.519323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.552177 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.552139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8" event={"ID":"acda8baf-5ef2-46bb-bb7a-90bed39b16ff","Type":"ContainerStarted","Data":"9b417cf493824dce827e70b4782c48f2f4fbb52421089443c0c5629dcce1e267"}
Jul 18 01:58:23.554515 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.554491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk" event={"ID":"350476df-fab6-46ec-ba57-27873edae6d7","Type":"ContainerStarted","Data":"7eec139b3ca9a0b077be603818e21770789526894c761595667d3ebed91d2d92"}
Jul 18 01:58:23.554628 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.554548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk" event={"ID":"350476df-fab6-46ec-ba57-27873edae6d7","Type":"ContainerStarted","Data":"c86ecdc5b24886fcd57cf93342f9eddd0495564039cc6273af3e570aee96cd38"}
Jul 18 01:58:23.554628 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.554571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk" event={"ID":"350476df-fab6-46ec-ba57-27873edae6d7","Type":"ContainerStarted","Data":"493b816277a0cd42678967683a0bf30d255d528fad3a7869f578541b0d09737d"}
Jul 18 01:58:23.620569 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-config-out\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620696 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620696 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620696 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-web-config\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620696 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620920 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620920 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvhxh\" (UniqueName: \"kubernetes.io/projected/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-kube-api-access-fvhxh\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620920 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620920 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.620920 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.621162 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-config-volume\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.621162 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.621162 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.620990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.622144 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.621828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.622535 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:23.622515    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-trusted-ca-bundle podName:c7ecc2b0-91e8-44d6-804b-1542e88a4a6b nodeName:}" failed. No retries permitted until 2026-07-18 01:58:24.122491943 +0000 UTC m=+78.615135769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "c7ecc2b0-91e8-44d6-804b-1542e88a4a6b") : configmap references non-existent config key: ca-bundle.crt
Jul 18 01:58:23.623140 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.623116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.625008 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.624989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-web-config\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.625440 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.625390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.626771 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.626750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.628758 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.628715    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-config-volume\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.628758 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.628744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.629627 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.629603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.630865 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.630843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.631389 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.631349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-config-out\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.631533 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.631514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:23.631588 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:23.631547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvhxh\" (UniqueName: \"kubernetes.io/projected/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-kube-api-access-fvhxh\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:24.126476 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.126445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:24.126668 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.126504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:24.127687 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.127659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ecc2b0-91e8-44d6-804b-1542e88a4a6b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:24.129347 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.129326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fe0498d-6e61-41c2-add3-ee8a1db91da4-node-exporter-tls\") pod \"node-exporter-55z2j\" (UID: \"0fe0498d-6e61-41c2-add3-ee8a1db91da4\") " pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:24.238761 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.238733    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-55z2j"
Jul 18 01:58:24.254997 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:24.254974    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe0498d_6e61_41c2_add3_ee8a1db91da4.slice/crio-7b106e32f61361c36c6048b7fba357f57d975a449d1667f4f5eda52c5504f3f3 WatchSource:0}: Error finding container 7b106e32f61361c36c6048b7fba357f57d975a449d1667f4f5eda52c5504f3f3: Status 404 returned error can't find the container with id 7b106e32f61361c36c6048b7fba357f57d975a449d1667f4f5eda52c5504f3f3
Jul 18 01:58:24.362469 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.362445    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:58:24.562262 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.562229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8" event={"ID":"acda8baf-5ef2-46bb-bb7a-90bed39b16ff","Type":"ContainerStarted","Data":"eab24fa6a4d666155f3b6f9cab42d9aa2a1460ce7ef08f4c3ed9789180cbf4a4"}
Jul 18 01:58:24.562741 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.562331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8" event={"ID":"acda8baf-5ef2-46bb-bb7a-90bed39b16ff","Type":"ContainerStarted","Data":"960b0be5afb605ca542c217e7b138157d5c6a5c661488dc862eedc255527fe42"}
Jul 18 01:58:24.562741 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.562367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8" event={"ID":"acda8baf-5ef2-46bb-bb7a-90bed39b16ff","Type":"ContainerStarted","Data":"4ceaaa2cf7eb905bd35e92ca3b56a9598ab9297445ac52f1697487fbf5e45d4c"}
Jul 18 01:58:24.565053 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.564933    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:58:24.565792 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:24.565712    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ecc2b0_91e8_44d6_804b_1542e88a4a6b.slice/crio-36e7f22bebf47d43e0dd6cb677b9e96b1fa2cfbbb2228479c4c43dd378db28fa WatchSource:0}: Error finding container 36e7f22bebf47d43e0dd6cb677b9e96b1fa2cfbbb2228479c4c43dd378db28fa: Status 404 returned error can't find the container with id 36e7f22bebf47d43e0dd6cb677b9e96b1fa2cfbbb2228479c4c43dd378db28fa
Jul 18 01:58:24.566169 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.566146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk" event={"ID":"350476df-fab6-46ec-ba57-27873edae6d7","Type":"ContainerStarted","Data":"f4fdff7de8ca0e2edf05191c81146e734a50fabcf2dc983c2d00a163a72707f7"}
Jul 18 01:58:24.567557 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.567426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-55z2j" event={"ID":"0fe0498d-6e61-41c2-add3-ee8a1db91da4","Type":"ContainerStarted","Data":"7b106e32f61361c36c6048b7fba357f57d975a449d1667f4f5eda52c5504f3f3"}
Jul 18 01:58:24.598081 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.598041    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5t5rk" podStartSLOduration=1.640641691 podStartE2EDuration="2.598025796s" totalImagesPullingTime="957.384105ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:24.597307428 +0000 UTC m=+79.089951256" watchObservedRunningTime="2026-07-18 01:58:24.598025796 +0000 UTC m=+79.090669626"
Jul 18 01:58:24.598917 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:24.598767    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-4pvj8" podStartSLOduration=1.18129522 podStartE2EDuration="2.598757984s" totalImagesPullingTime="1.417462764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:24.581025486 +0000 UTC m=+79.073669310" watchObservedRunningTime="2026-07-18 01:58:24.598757984 +0000 UTC m=+79.091401810"
Jul 18 01:58:25.358805 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.358726    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-699ff677f6-mk99f"]
Jul 18 01:58:25.362354 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.362325    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.365061 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365037    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 18 01:58:25.365183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365037    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 18 01:58:25.365183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365079    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 18 01:58:25.365183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365094    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-v2glw\""
Jul 18 01:58:25.365183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365106    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 18 01:58:25.365183 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365147    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a879vkb8ih1gn\""
Jul 18 01:58:25.365420 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.365408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 18 01:58:25.373341 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.373317    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-699ff677f6-mk99f"]
Jul 18 01:58:25.541049 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541259 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541259 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-tls\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541259 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541259 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-grpc-tls\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541259 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39033c8c-819c-412c-a159-f81ab0e3b072-metrics-client-ca\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541463 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqpx\" (UniqueName: \"kubernetes.io/projected/39033c8c-819c-412c-a159-f81ab0e3b072-kube-api-access-4jqpx\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.541463 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.541294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.573072 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.573039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"36e7f22bebf47d43e0dd6cb677b9e96b1fa2cfbbb2228479c4c43dd378db28fa"}
Jul 18 01:58:25.574942 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.574916    2576 generic.go:356] "Generic (PLEG): container finished" podID="0fe0498d-6e61-41c2-add3-ee8a1db91da4" containerID="19a697628bf563595865df852a445f6409825370b038e27f234c409a66d8fe69" exitCode=0
Jul 18 01:58:25.575069 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.574984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-55z2j" event={"ID":"0fe0498d-6e61-41c2-add3-ee8a1db91da4","Type":"ContainerDied","Data":"19a697628bf563595865df852a445f6409825370b038e27f234c409a66d8fe69"}
Jul 18 01:58:25.642023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.641952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqpx\" (UniqueName: \"kubernetes.io/projected/39033c8c-819c-412c-a159-f81ab0e3b072-kube-api-access-4jqpx\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642023 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.641994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642331 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642331 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642331 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-tls\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642331 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642331 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-grpc-tls\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642331 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39033c8c-819c-412c-a159-f81ab0e3b072-metrics-client-ca\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.642967 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.642945    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39033c8c-819c-412c-a159-f81ab0e3b072-metrics-client-ca\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.645613 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.645587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.645706 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.645640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.645706 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.645694    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.645828 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.645727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-tls\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.645828 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.645811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-grpc-tls\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.645931 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.645845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/39033c8c-819c-412c-a159-f81ab0e3b072-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.649953 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.649933    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqpx\" (UniqueName: \"kubernetes.io/projected/39033c8c-819c-412c-a159-f81ab0e3b072-kube-api-access-4jqpx\") pod \"thanos-querier-699ff677f6-mk99f\" (UID: \"39033c8c-819c-412c-a159-f81ab0e3b072\") " pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.671872 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.671852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:25.883233 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:25.883186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-699ff677f6-mk99f"]
Jul 18 01:58:25.928334 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:25.928311    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39033c8c_819c_412c_a159_f81ab0e3b072.slice/crio-3658efca7804fd17546a51eded283bf8444c8e84186df95ce404bb2e69dfd5c1 WatchSource:0}: Error finding container 3658efca7804fd17546a51eded283bf8444c8e84186df95ce404bb2e69dfd5c1: Status 404 returned error can't find the container with id 3658efca7804fd17546a51eded283bf8444c8e84186df95ce404bb2e69dfd5c1
Jul 18 01:58:26.580051 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.580014    2576 generic.go:356] "Generic (PLEG): container finished" podID="c7ecc2b0-91e8-44d6-804b-1542e88a4a6b" containerID="f52b79db3c82c523e007d0c7deefc4660b9940c8c1fcd23f4f716f5aae1780b8" exitCode=0
Jul 18 01:58:26.580514 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.580102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerDied","Data":"f52b79db3c82c523e007d0c7deefc4660b9940c8c1fcd23f4f716f5aae1780b8"}
Jul 18 01:58:26.581391 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.581361    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"3658efca7804fd17546a51eded283bf8444c8e84186df95ce404bb2e69dfd5c1"}
Jul 18 01:58:26.583727 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.583702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-55z2j" event={"ID":"0fe0498d-6e61-41c2-add3-ee8a1db91da4","Type":"ContainerStarted","Data":"f522502402446ac3ca0d4eedc08e0e4863a3039b0003561663495acd3ee48b20"}
Jul 18 01:58:26.583835 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.583753    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-55z2j" event={"ID":"0fe0498d-6e61-41c2-add3-ee8a1db91da4","Type":"ContainerStarted","Data":"0d0c2ca4e72366541f60af9d95b387cd22bd963a2819f905a73d267ac2dbd062"}
Jul 18 01:58:26.623202 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.623156    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-55z2j" podStartSLOduration=3.754810264 podStartE2EDuration="4.623141804s" totalImagesPullingTime="804.667178ms" totalInitContainerRuntime="63.664362ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:26.622266585 +0000 UTC m=+81.114910413" watchObservedRunningTime="2026-07-18 01:58:26.623141804 +0000 UTC m=+81.115785629"
Jul 18 01:58:26.768998 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.768974    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-678c97459c-j4ctt"]
Jul 18 01:58:26.771418 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.771399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.773991 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.773967    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 18 01:58:26.773991 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.773986    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fh3sah581lpjm\""
Jul 18 01:58:26.774130 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.774046    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 18 01:58:26.774130 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.774100    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hmrbv\""
Jul 18 01:58:26.774361 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.774344    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 18 01:58:26.774446 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.774400    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 18 01:58:26.785358 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.785338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-678c97459c-j4ctt"]
Jul 18 01:58:26.951564 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.951714 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-secret-metrics-server-client-certs\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.951714 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-client-ca-bundle\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.951714 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mxg\" (UniqueName: \"kubernetes.io/projected/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-kube-api-access-n4mxg\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.951714 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-metrics-server-audit-profiles\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.951909 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-audit-log\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:26.951909 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:26.951807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-secret-metrics-server-tls\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052592 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-secret-metrics-server-tls\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052728 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052728 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-secret-metrics-server-client-certs\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052728 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-client-ca-bundle\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052728 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mxg\" (UniqueName: \"kubernetes.io/projected/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-kube-api-access-n4mxg\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052930 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-metrics-server-audit-profiles\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.052930 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.052787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-audit-log\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.053889 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.053856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.054185 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.054136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-metrics-server-audit-profiles\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.054408 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.054383    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-audit-log\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.055471 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.055452    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-client-ca-bundle\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.055552 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.055476    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-secret-metrics-server-tls\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.055606 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.055547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-secret-metrics-server-client-certs\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.060663 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.060641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mxg\" (UniqueName: \"kubernetes.io/projected/cdc861e9-c03e-4ae9-94e3-a16f4743df8f-kube-api-access-n4mxg\") pod \"metrics-server-678c97459c-j4ctt\" (UID: \"cdc861e9-c03e-4ae9-94e3-a16f4743df8f\") " pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.082693 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.082671    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:27.160689 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.160606    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"]
Jul 18 01:58:27.163378 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.163356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.166248 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.166229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 18 01:58:27.166525 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.166506    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fn8lf\""
Jul 18 01:58:27.178918 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.178895    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75d568c9dd-zhrss"]
Jul 18 01:58:27.189703 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.189647    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"]
Jul 18 01:58:27.249681 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.249609    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c89c8fcdc-79jz2"]
Jul 18 01:58:27.252333 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.252309    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.255241 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.255214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.255391 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.255278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgcbb\" (UniqueName: \"kubernetes.io/projected/5ba1b558-ac73-4533-b25d-62e925dca452-kube-api-access-xgcbb\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.262303 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.262277    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-678c97459c-j4ctt"]
Jul 18 01:58:27.269540 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.267138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c89c8fcdc-79jz2"]
Jul 18 01:58:27.270463 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:27.270441    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc861e9_c03e_4ae9_94e3_a16f4743df8f.slice/crio-33857eb64a535507e4ba355d6193fd546d183d420aa43cc8ec8d041aaf12c6ec WatchSource:0}: Error finding container 33857eb64a535507e4ba355d6193fd546d183d420aa43cc8ec8d041aaf12c6ec: Status 404 returned error can't find the container with id 33857eb64a535507e4ba355d6193fd546d183d420aa43cc8ec8d041aaf12c6ec
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpm6v\" (UniqueName: \"kubernetes.io/projected/17dd6654-c8da-40db-8466-ade4bf588fed-kube-api-access-qpm6v\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356526    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-service-ca\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-trusted-ca-bundle\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-serving-cert\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:27.356629    2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-oauth-serving-cert\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgcbb\" (UniqueName: \"kubernetes.io/projected/5ba1b558-ac73-4533-b25d-62e925dca452-kube-api-access-xgcbb\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:27.356695    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert podName:5ba1b558-ac73-4533-b25d-62e925dca452 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:27.856673718 +0000 UTC m=+82.349317535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-bhbzk" (UID: "5ba1b558-ac73-4533-b25d-62e925dca452") : secret "monitoring-plugin-cert" not found
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-oauth-config\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.356985 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.356819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-console-config\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.365796 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.365775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgcbb\" (UniqueName: \"kubernetes.io/projected/5ba1b558-ac73-4533-b25d-62e925dca452-kube-api-access-xgcbb\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.458109 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-service-ca\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.458311 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-trusted-ca-bundle\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.458311 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-serving-cert\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.458311 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-oauth-serving-cert\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.458311 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-oauth-config\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.458538 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-console-config\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.458538 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.458388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpm6v\" (UniqueName: \"kubernetes.io/projected/17dd6654-c8da-40db-8466-ade4bf588fed-kube-api-access-qpm6v\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.459577 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.459275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-trusted-ca-bundle\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.459577 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.459342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-service-ca\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.459577 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.459458    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-console-config\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.461049 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.460826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-oauth-serving-cert\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.461782 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.461738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-serving-cert\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.462548 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.462511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-oauth-config\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.467360 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.467334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpm6v\" (UniqueName: \"kubernetes.io/projected/17dd6654-c8da-40db-8466-ade4bf588fed-kube-api-access-qpm6v\") pod \"console-7c89c8fcdc-79jz2\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") " pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.569937 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.569848    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:27.590336 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.590295    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt" event={"ID":"cdc861e9-c03e-4ae9-94e3-a16f4743df8f","Type":"ContainerStarted","Data":"33857eb64a535507e4ba355d6193fd546d183d420aa43cc8ec8d041aaf12c6ec"}
Jul 18 01:58:27.717945 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.717918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c89c8fcdc-79jz2"]
Jul 18 01:58:27.862118 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:27.862041    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:27.862303 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:27.862217    2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 18 01:58:27.862367 ip-10-0-133-113 kubenswrapper[2576]: E0718 01:58:27.862303    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert podName:5ba1b558-ac73-4533-b25d-62e925dca452 nodeName:}" failed. No retries permitted until 2026-07-18 01:58:28.862281789 +0000 UTC m=+83.354925597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-bhbzk" (UID: "5ba1b558-ac73-4533-b25d-62e925dca452") : secret "monitoring-plugin-cert" not found
Jul 18 01:58:28.152943 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:28.152771    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17dd6654_c8da_40db_8466_ade4bf588fed.slice/crio-c9e1467210837e36fb011dd6d52820ef054be1bb2f86ed81c9a5b0fe107f4731 WatchSource:0}: Error finding container c9e1467210837e36fb011dd6d52820ef054be1bb2f86ed81c9a5b0fe107f4731: Status 404 returned error can't find the container with id c9e1467210837e36fb011dd6d52820ef054be1bb2f86ed81c9a5b0fe107f4731
Jul 18 01:58:28.571466 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.569752    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:58:28.576547 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.576484    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.579035 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579017    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 01:58:28.579250 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579234    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 01:58:28.579357 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579342    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 01:58:28.579420 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579358    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 01:58:28.579554 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579542    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 01:58:28.579595 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 01:58:28.579974 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.579955    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 01:58:28.580319 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.580300    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 01:58:28.580319 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.580311    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 01:58:28.580475 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.580351    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 01:58:28.580475 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.580316    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 01:58:28.582494 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.582466    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 01:58:28.582901 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.582883    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4k9cv\""
Jul 18 01:58:28.582990 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.582886    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-lc00rq61fua9\""
Jul 18 01:58:28.582990 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.582908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 01:58:28.584220 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.584203    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 01:58:28.590069 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.590027    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:58:28.597932 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.597896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c89c8fcdc-79jz2" event={"ID":"17dd6654-c8da-40db-8466-ade4bf588fed","Type":"ContainerStarted","Data":"c9e1467210837e36fb011dd6d52820ef054be1bb2f86ed81c9a5b0fe107f4731"}
Jul 18 01:58:28.668936 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.668886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669063 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.668958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669063 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-config\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669176 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669176 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669322 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669322 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669322 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669474 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669474 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669474 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-config-out\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669474 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669673 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669673 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669673 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669553    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669673 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669878 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669878 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.669979 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnw4b\" (UniqueName: \"kubernetes.io/projected/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-kube-api-access-jnw4b\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.670031 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.669982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-web-config\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770660    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-config-out\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770796    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.770982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.773740 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnw4b\" (UniqueName: \"kubernetes.io/projected/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-kube-api-access-jnw4b\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-web-config\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-config\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.771279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.772033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.772790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.773085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.774539 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.773855    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.777490 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.776495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.777490 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.776895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-web-config\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.778313 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.777817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.778313 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.777994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-config-out\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.779298 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.778967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.779622 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.779592    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.780091 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.780066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.782134 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.782085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.782935 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.782902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.791988 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.791963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnw4b\" (UniqueName: \"kubernetes.io/projected/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-kube-api-access-jnw4b\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.792082 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.792016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.794866 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.794819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.795049 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.795023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-config\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.795622 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.795598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.795826 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.795802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.797067 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.797042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6d1a2a57-a2ab-40d4-a6f9-f6e030165821-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6d1a2a57-a2ab-40d4-a6f9-f6e030165821\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:28.872784 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.871878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:28.875329 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.875280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5ba1b558-ac73-4533-b25d-62e925dca452-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bhbzk\" (UID: \"5ba1b558-ac73-4533-b25d-62e925dca452\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:28.975991 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.975967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:28.998997 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:28.998672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:29.152729 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.152690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"]
Jul 18 01:58:29.158618 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:29.158557    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba1b558_ac73_4533_b25d_62e925dca452.slice/crio-91e88dbd117257144186fcbf838494e4574849d0a8887cc938716b975f539123 WatchSource:0}: Error finding container 91e88dbd117257144186fcbf838494e4574849d0a8887cc938716b975f539123: Status 404 returned error can't find the container with id 91e88dbd117257144186fcbf838494e4574849d0a8887cc938716b975f539123
Jul 18 01:58:29.188343 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.188236    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:58:29.190463 ip-10-0-133-113 kubenswrapper[2576]: W0718 01:58:29.190439    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1a2a57_a2ab_40d4_a6f9_f6e030165821.slice/crio-bcc624995f719fdf1cf0085da47da6dd2fdc4428b6119031b486e30ec4dcb377 WatchSource:0}: Error finding container bcc624995f719fdf1cf0085da47da6dd2fdc4428b6119031b486e30ec4dcb377: Status 404 returned error can't find the container with id bcc624995f719fdf1cf0085da47da6dd2fdc4428b6119031b486e30ec4dcb377
Jul 18 01:58:29.530958 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.530925    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:29.604895 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.604807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c89c8fcdc-79jz2" event={"ID":"17dd6654-c8da-40db-8466-ade4bf588fed","Type":"ContainerStarted","Data":"e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b"}
Jul 18 01:58:29.606775 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.606750    2576 generic.go:356] "Generic (PLEG): container finished" podID="6d1a2a57-a2ab-40d4-a6f9-f6e030165821" containerID="af07010c728a317a5b77685c72f1b67da93332fec682500de626229e9ac98545" exitCode=0
Jul 18 01:58:29.606893 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.606834    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerDied","Data":"af07010c728a317a5b77685c72f1b67da93332fec682500de626229e9ac98545"}
Jul 18 01:58:29.606953 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.606906    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"bcc624995f719fdf1cf0085da47da6dd2fdc4428b6119031b486e30ec4dcb377"}
Jul 18 01:58:29.610502 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.610484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"ee0d5d1337be00baee6f402825b5f0bcfe67eb38640be5bb65333907ba683763"}
Jul 18 01:58:29.610608 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.610547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"d052812b7b39a0cd916196100244622bce5e73ecaa840423ff993f833842420f"}
Jul 18 01:58:29.610608 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.610592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"d4fee6be210f9838224fc9559aafe26f3d750ced9984098eee14bee49241cbaa"}
Jul 18 01:58:29.610723 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.610630    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"1bf39974e5947d580998ffac96d56ba63205b743f567b9bc08e9ec47a77c2c33"}
Jul 18 01:58:29.610723 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.610669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"3e09c83724b57c2b02ee4be7f8280778bfbfd10e5ec43843bc982941ac7f420d"}
Jul 18 01:58:29.612839 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.612816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"bf231f409d9e7df4b0823f86bd13adaba493554651065a13a1dbb0448b7f8192"}
Jul 18 01:58:29.612941 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.612875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"1ea7651d15935cc9c3fd990ee98424926ed3be3f4b4a58524092e8947cdcbbeb"}
Jul 18 01:58:29.612941 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.612908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"efade5116679a04e35a93f2be94edd0d3e8fd7055970ab0b0351777413aab8b3"}
Jul 18 01:58:29.614096 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.614076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk" event={"ID":"5ba1b558-ac73-4533-b25d-62e925dca452","Type":"ContainerStarted","Data":"91e88dbd117257144186fcbf838494e4574849d0a8887cc938716b975f539123"}
Jul 18 01:58:29.624959 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:29.624920    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c89c8fcdc-79jz2" podStartSLOduration=2.624907997 podStartE2EDuration="2.624907997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:29.62358305 +0000 UTC m=+84.116226876" watchObservedRunningTime="2026-07-18 01:58:29.624907997 +0000 UTC m=+84.117551868"
Jul 18 01:58:30.623620 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.623586    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7ecc2b0-91e8-44d6-804b-1542e88a4a6b","Type":"ContainerStarted","Data":"e4232c7cdef7f40e7e50893271a998e8addae4f38cc2aecc83f9cd77a97c6553"}
Jul 18 01:58:30.627822 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.627798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"efa3077c4fdcc70b4c72857efde541ad45a2be0091cc4c549120d466cb7e0f64"}
Jul 18 01:58:30.628060 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.628025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:30.628138 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.628077    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"a106e033ee0c254a736a1c8a1ccd6336b987569b127dbac4e87c024c08e39516"}
Jul 18 01:58:30.628138 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.628119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" event={"ID":"39033c8c-819c-412c-a159-f81ab0e3b072","Type":"ContainerStarted","Data":"354f073711a787c3d7813de4747b74428a79cccb57a42ad66398c7cb2f460cb3"}
Jul 18 01:58:30.630715 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.630690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt" event={"ID":"cdc861e9-c03e-4ae9-94e3-a16f4743df8f","Type":"ContainerStarted","Data":"a4a7d982badc9ccbd796ff62347c1876eea40fd8a465705f0f7a268ff21d2d51"}
Jul 18 01:58:30.650825 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.650747    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.59326584 podStartE2EDuration="7.650734939s" totalImagesPullingTime="4.036302043s" totalInitContainerRuntime="21.167056ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:30.648611514 +0000 UTC m=+85.141255342" watchObservedRunningTime="2026-07-18 01:58:30.650734939 +0000 UTC m=+85.143378765"
Jul 18 01:58:30.693230 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.691622    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f" podStartSLOduration=2.103415481 podStartE2EDuration="5.691603671s" totalImagesPullingTime="3.58818819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:30.672558151 +0000 UTC m=+85.165201978" watchObservedRunningTime="2026-07-18 01:58:30.691603671 +0000 UTC m=+85.184247497"
Jul 18 01:58:30.693230 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:30.691889    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt" podStartSLOduration=2.088414094 podStartE2EDuration="4.691877402s" totalImagesPullingTime="2.603463308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:30.691092575 +0000 UTC m=+85.183736400" watchObservedRunningTime="2026-07-18 01:58:30.691877402 +0000 UTC m=+85.184521227"
Jul 18 01:58:31.636344 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:31.636304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk" event={"ID":"5ba1b558-ac73-4533-b25d-62e925dca452","Type":"ContainerStarted","Data":"bef9ac4e37196cc386f8eefc665bcd78f666246447592ccf1821470aa5025f80"}
Jul 18 01:58:31.651213 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:31.651154    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk" podStartSLOduration=2.965041233 podStartE2EDuration="4.651138445s" totalImagesPullingTime="1.686097212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:31.650792583 +0000 UTC m=+86.143436410" watchObservedRunningTime="2026-07-18 01:58:31.651138445 +0000 UTC m=+86.143782271"
Jul 18 01:58:32.641063 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:32.641028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:32.647106 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:32.647080    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bhbzk"
Jul 18 01:58:33.650249 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.650209    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"49b89f5c19b202e2803ebf9e71048833810f682c443e05d7cb4042e602eba5fc"}
Jul 18 01:58:33.650697 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.650284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"d32db07e6c9fd98939455760363b005d1d478f0383dbd7c5bfea16852bf4fe2d"}
Jul 18 01:58:33.650697 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.650314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"9b54708908b40224ac99abce7f6fd48e810d09de411e1869f15fcc4e358811b2"}
Jul 18 01:58:33.650697 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.650341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"6472bb4e6952642542b2196ed295afb4ccf276179f5cb420db070667efb0e3d5"}
Jul 18 01:58:33.650697 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.650371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"ade94e270ac2434f26ea2e31e271d04ae36956b998cbbbd7cde6308d6c914f72"}
Jul 18 01:58:33.650697 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.650395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6d1a2a57-a2ab-40d4-a6f9-f6e030165821","Type":"ContainerStarted","Data":"f4e8f300a37570bb83b43ddc2d7bbee1ddac413885b0fba598f5a577fece3e91"}
Jul 18 01:58:33.678578 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.678528    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.408254071 podStartE2EDuration="5.678507738s" totalImagesPullingTime="3.246208564s" totalInitContainerRuntime="24.045103ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:33.676266041 +0000 UTC m=+88.168909870" watchObservedRunningTime="2026-07-18 01:58:33.678507738 +0000 UTC m=+88.171151566"
Jul 18 01:58:33.999072 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:33.999043    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:36.642502 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:36.642472    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-699ff677f6-mk99f"
Jul 18 01:58:37.571524 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:37.571490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:37.571688 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:37.571562    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:37.575775 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:37.575753    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:37.674328 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:37.674294    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 01:58:47.083212 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:47.083163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:47.083609 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:47.083232    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:58:49.118138 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:49.118094    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mtpxd_dfc9fb34-b2f3-467a-ae70-77afade7a241/serve-healthcheck-canary/0.log"
Jul 18 01:58:51.544695 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:51.544666    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wqbsp"
Jul 18 01:58:52.209598 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:52.209543    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75d568c9dd-zhrss" podUID="ee320774-08ef-47a5-bf97-53a80ebd01e2" containerName="console" containerID="cri-o://aa06f5d33cc46a2e5a6fa2e67ce65545ca29580827fd26bfc4b212b508a4db17" gracePeriod=15
Jul 18 01:58:52.728018 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:52.727993    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee320774-08ef-47a5-bf97-53a80ebd01e2" containerID="aa06f5d33cc46a2e5a6fa2e67ce65545ca29580827fd26bfc4b212b508a4db17" exitCode=0
Jul 18 01:58:52.728365 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:52.728075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d568c9dd-zhrss" event={"ID":"ee320774-08ef-47a5-bf97-53a80ebd01e2","Type":"ContainerDied","Data":"aa06f5d33cc46a2e5a6fa2e67ce65545ca29580827fd26bfc4b212b508a4db17"}
Jul 18 01:58:52.975364 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:52.975340    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:53.067137 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067109    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkzmn\" (UniqueName: \"kubernetes.io/projected/ee320774-08ef-47a5-bf97-53a80ebd01e2-kube-api-access-vkzmn\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067295 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067159    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-oauth-serving-cert\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067295 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067179    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-trusted-ca-bundle\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067295 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-serving-cert\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067295 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-service-ca\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067520 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067301    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-oauth-config\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067520 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067327    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-config\") pod \"ee320774-08ef-47a5-bf97-53a80ebd01e2\" (UID: \"ee320774-08ef-47a5-bf97-53a80ebd01e2\") "
Jul 18 01:58:53.067672 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:58:53.067734 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:58:53.067790 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:58:53.067891 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.067861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-config" (OuterVolumeSpecName: "console-config") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:58:53.069549 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.069508    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee320774-08ef-47a5-bf97-53a80ebd01e2-kube-api-access-vkzmn" (OuterVolumeSpecName: "kube-api-access-vkzmn") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "kube-api-access-vkzmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:58:53.069628 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.069512    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:58:53.069628 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.069573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ee320774-08ef-47a5-bf97-53a80ebd01e2" (UID: "ee320774-08ef-47a5-bf97-53a80ebd01e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:58:53.168891 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168837    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-trusted-ca-bundle\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.168891 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168858    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-serving-cert\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.168891 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168867    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.168891 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168876    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-oauth-config\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.168891 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168886    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-console-config\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.169096 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168911    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkzmn\" (UniqueName: \"kubernetes.io/projected/ee320774-08ef-47a5-bf97-53a80ebd01e2-kube-api-access-vkzmn\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.169096 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.168920    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee320774-08ef-47a5-bf97-53a80ebd01e2-oauth-serving-cert\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 01:58:53.733116 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.733084    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d568c9dd-zhrss"
Jul 18 01:58:53.733116 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.733094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d568c9dd-zhrss" event={"ID":"ee320774-08ef-47a5-bf97-53a80ebd01e2","Type":"ContainerDied","Data":"5b18861c8dcda4fa13fcba8c6f143a63af85be86ed4d5d4358fb7fa0d914e48d"}
Jul 18 01:58:53.733639 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.733154    2576 scope.go:117] "RemoveContainer" containerID="aa06f5d33cc46a2e5a6fa2e67ce65545ca29580827fd26bfc4b212b508a4db17"
Jul 18 01:58:53.750344 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.750320    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75d568c9dd-zhrss"]
Jul 18 01:58:53.753722 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:53.753702    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75d568c9dd-zhrss"]
Jul 18 01:58:55.080240 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:58:55.080209    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee320774-08ef-47a5-bf97-53a80ebd01e2" path="/var/lib/kubelet/pods/ee320774-08ef-47a5-bf97-53a80ebd01e2/volumes"
Jul 18 01:59:07.089723 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:59:07.089691    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:59:07.093705 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:59:07.093683    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-678c97459c-j4ctt"
Jul 18 01:59:28.999535 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:59:28.999500    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:59:29.019235 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:59:29.019186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:59:29.900017 ip-10-0-133-113 kubenswrapper[2576]: I0718 01:59:29.899991    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 02:00:02.583353 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:02.583271    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c89c8fcdc-79jz2"]
Jul 18 02:00:27.604539 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:27.604482    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c89c8fcdc-79jz2" podUID="17dd6654-c8da-40db-8466-ade4bf588fed" containerName="console" containerID="cri-o://e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b" gracePeriod=15
Jul 18 02:00:27.669475 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:27.669446    2576 patch_prober.go:28] interesting pod/console-7c89c8fcdc-79jz2 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body=
Jul 18 02:00:27.669570 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:27.669525    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-7c89c8fcdc-79jz2" podUID="17dd6654-c8da-40db-8466-ade4bf588fed" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused"
Jul 18 02:00:27.848829 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:27.848807    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 02:00:28.010348 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010324    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-oauth-serving-cert\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010489 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010363    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-trusted-ca-bundle\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010489 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010409    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-serving-cert\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010489 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010452    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-service-ca\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010489 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010487    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-oauth-config\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010698 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010535    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpm6v\" (UniqueName: \"kubernetes.io/projected/17dd6654-c8da-40db-8466-ade4bf588fed-kube-api-access-qpm6v\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010698 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010564    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-console-config\") pod \"17dd6654-c8da-40db-8466-ade4bf588fed\" (UID: \"17dd6654-c8da-40db-8466-ade4bf588fed\") "
Jul 18 02:00:28.010868 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010839    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:00:28.010942 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010908    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-service-ca" (OuterVolumeSpecName: "service-ca") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:00:28.010997 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010941    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:00:28.010997 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.010977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-console-config" (OuterVolumeSpecName: "console-config") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:00:28.012717 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.012689    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:00:28.012825 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.012740    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:00:28.012825 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.012784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17dd6654-c8da-40db-8466-ade4bf588fed-kube-api-access-qpm6v" (OuterVolumeSpecName: "kube-api-access-qpm6v") pod "17dd6654-c8da-40db-8466-ade4bf588fed" (UID: "17dd6654-c8da-40db-8466-ade4bf588fed"). InnerVolumeSpecName "kube-api-access-qpm6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:00:28.110340 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.110312    2576 generic.go:356] "Generic (PLEG): container finished" podID="17dd6654-c8da-40db-8466-ade4bf588fed" containerID="e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b" exitCode=0
Jul 18 02:00:28.110458 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.110354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c89c8fcdc-79jz2" event={"ID":"17dd6654-c8da-40db-8466-ade4bf588fed","Type":"ContainerDied","Data":"e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b"}
Jul 18 02:00:28.110458 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.110376    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c89c8fcdc-79jz2"
Jul 18 02:00:28.110458 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.110385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c89c8fcdc-79jz2" event={"ID":"17dd6654-c8da-40db-8466-ade4bf588fed","Type":"ContainerDied","Data":"c9e1467210837e36fb011dd6d52820ef054be1bb2f86ed81c9a5b0fe107f4731"}
Jul 18 02:00:28.110458 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.110404    2576 scope.go:117] "RemoveContainer" containerID="e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b"
Jul 18 02:00:28.111316 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111293    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-console-config\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.111316 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111316    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-oauth-serving-cert\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.111461 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111330    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-trusted-ca-bundle\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.111461 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111339    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-serving-cert\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.111461 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111347    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17dd6654-c8da-40db-8466-ade4bf588fed-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.111461 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111363    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17dd6654-c8da-40db-8466-ade4bf588fed-console-oauth-config\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.111461 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.111371    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpm6v\" (UniqueName: \"kubernetes.io/projected/17dd6654-c8da-40db-8466-ade4bf588fed-kube-api-access-qpm6v\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:00:28.119472 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.119455    2576 scope.go:117] "RemoveContainer" containerID="e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b"
Jul 18 02:00:28.119730 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:00:28.119710    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b\": container with ID starting with e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b not found: ID does not exist" containerID="e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b"
Jul 18 02:00:28.119789 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.119738    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b"} err="failed to get container status \"e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b\": rpc error: code = NotFound desc = could not find container \"e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b\": container with ID starting with e6e75fdfdd788c203ca45709ad1ce70ff872de02f334bd2066f05bb3e75e829b not found: ID does not exist"
Jul 18 02:00:28.133056 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.133027    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c89c8fcdc-79jz2"]
Jul 18 02:00:28.135127 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:28.135108    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c89c8fcdc-79jz2"]
Jul 18 02:00:29.080005 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:00:29.079971    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17dd6654-c8da-40db-8466-ade4bf588fed" path="/var/lib/kubelet/pods/17dd6654-c8da-40db-8466-ade4bf588fed/volumes"
Jul 18 02:01:16.359001 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.358964    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"]
Jul 18 02:01:16.359482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.359349    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee320774-08ef-47a5-bf97-53a80ebd01e2" containerName="console"
Jul 18 02:01:16.359482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.359363    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee320774-08ef-47a5-bf97-53a80ebd01e2" containerName="console"
Jul 18 02:01:16.359482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.359375    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17dd6654-c8da-40db-8466-ade4bf588fed" containerName="console"
Jul 18 02:01:16.359482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.359380    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="17dd6654-c8da-40db-8466-ade4bf588fed" containerName="console"
Jul 18 02:01:16.359482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.359435    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="17dd6654-c8da-40db-8466-ade4bf588fed" containerName="console"
Jul 18 02:01:16.359482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.359446    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee320774-08ef-47a5-bf97-53a80ebd01e2" containerName="console"
Jul 18 02:01:16.362058 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.362039    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.364705 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.364677    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 18 02:01:16.364794 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.364781    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 18 02:01:16.365914 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.365896    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2jhvc\""
Jul 18 02:01:16.371102 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.371079    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"]
Jul 18 02:01:16.469223 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.469179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/026a3176-5a72-465b-a0b8-92cab3165ec4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7kxkn\" (UID: \"026a3176-5a72-465b-a0b8-92cab3165ec4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.469377 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.469266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9wrm\" (UniqueName: \"kubernetes.io/projected/026a3176-5a72-465b-a0b8-92cab3165ec4-kube-api-access-l9wrm\") pod \"cert-manager-cainjector-559cdd8c87-7kxkn\" (UID: \"026a3176-5a72-465b-a0b8-92cab3165ec4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.570444 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.570420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9wrm\" (UniqueName: \"kubernetes.io/projected/026a3176-5a72-465b-a0b8-92cab3165ec4-kube-api-access-l9wrm\") pod \"cert-manager-cainjector-559cdd8c87-7kxkn\" (UID: \"026a3176-5a72-465b-a0b8-92cab3165ec4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.570551 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.570467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/026a3176-5a72-465b-a0b8-92cab3165ec4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7kxkn\" (UID: \"026a3176-5a72-465b-a0b8-92cab3165ec4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.579819 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.579787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/026a3176-5a72-465b-a0b8-92cab3165ec4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7kxkn\" (UID: \"026a3176-5a72-465b-a0b8-92cab3165ec4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.579913 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.579904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9wrm\" (UniqueName: \"kubernetes.io/projected/026a3176-5a72-465b-a0b8-92cab3165ec4-kube-api-access-l9wrm\") pod \"cert-manager-cainjector-559cdd8c87-7kxkn\" (UID: \"026a3176-5a72-465b-a0b8-92cab3165ec4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.683094 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.683033    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"
Jul 18 02:01:16.803288 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:16.803244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn"]
Jul 18 02:01:16.805651 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:01:16.805619    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod026a3176_5a72_465b_a0b8_92cab3165ec4.slice/crio-d8f6e29b693e8ce492e658b5cdc00663bc641fa9209dd51e765fd359424ed627 WatchSource:0}: Error finding container d8f6e29b693e8ce492e658b5cdc00663bc641fa9209dd51e765fd359424ed627: Status 404 returned error can't find the container with id d8f6e29b693e8ce492e658b5cdc00663bc641fa9209dd51e765fd359424ed627
Jul 18 02:01:17.305921 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:17.305883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn" event={"ID":"026a3176-5a72-465b-a0b8-92cab3165ec4","Type":"ContainerStarted","Data":"d8f6e29b693e8ce492e658b5cdc00663bc641fa9209dd51e765fd359424ed627"}
Jul 18 02:01:20.322626 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:20.322533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn" event={"ID":"026a3176-5a72-465b-a0b8-92cab3165ec4","Type":"ContainerStarted","Data":"2becb697da9d707798fc086b246e29c6f8a14906a3fb031e2ba72f6004256d54"}
Jul 18 02:01:20.338635 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:20.338583    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7kxkn" podStartSLOduration=1.112155894 podStartE2EDuration="4.338565908s" totalImagesPullingTime="3.226410014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:20.337969562 +0000 UTC m=+254.830613388" watchObservedRunningTime="2026-07-18 02:01:20.338565908 +0000 UTC m=+254.831209733"
Jul 18 02:01:25.381553 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.381525    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rp2lv"]
Jul 18 02:01:25.384234 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.384187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.386718 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.386699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-c2f79\""
Jul 18 02:01:25.394657 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.394635    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rp2lv"]
Jul 18 02:01:25.445598 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.445573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcbgg\" (UniqueName: \"kubernetes.io/projected/3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74-kube-api-access-gcbgg\") pod \"cert-manager-55b4d954d5-rp2lv\" (UID: \"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74\") " pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.445681 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.445619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74-bound-sa-token\") pod \"cert-manager-55b4d954d5-rp2lv\" (UID: \"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74\") " pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.546882 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.546852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74-bound-sa-token\") pod \"cert-manager-55b4d954d5-rp2lv\" (UID: \"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74\") " pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.547009 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.546947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcbgg\" (UniqueName: \"kubernetes.io/projected/3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74-kube-api-access-gcbgg\") pod \"cert-manager-55b4d954d5-rp2lv\" (UID: \"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74\") " pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.555021 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.554995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74-bound-sa-token\") pod \"cert-manager-55b4d954d5-rp2lv\" (UID: \"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74\") " pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.555129 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.555078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcbgg\" (UniqueName: \"kubernetes.io/projected/3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74-kube-api-access-gcbgg\") pod \"cert-manager-55b4d954d5-rp2lv\" (UID: \"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74\") " pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.693301 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.693276    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rp2lv"
Jul 18 02:01:25.814765 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:25.814739    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rp2lv"]
Jul 18 02:01:25.817097 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:01:25.817071    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb269eb_6cd8_4ba5_bee9_56a1f2b49f74.slice/crio-595c4ae9e232585afcfce7e2ce5525d6283e984c765db52e2dcb801502f9cfcb WatchSource:0}: Error finding container 595c4ae9e232585afcfce7e2ce5525d6283e984c765db52e2dcb801502f9cfcb: Status 404 returned error can't find the container with id 595c4ae9e232585afcfce7e2ce5525d6283e984c765db52e2dcb801502f9cfcb
Jul 18 02:01:26.349396 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:26.349361    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rp2lv" event={"ID":"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74","Type":"ContainerStarted","Data":"5d2d721d0c1dd601cda50db90caf33778f07d2dd788b0671c6857c5cb0549049"}
Jul 18 02:01:26.349541 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:26.349404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rp2lv" event={"ID":"3fb269eb-6cd8-4ba5-bee9-56a1f2b49f74","Type":"ContainerStarted","Data":"595c4ae9e232585afcfce7e2ce5525d6283e984c765db52e2dcb801502f9cfcb"}
Jul 18 02:01:26.365881 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:26.365833    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rp2lv" podStartSLOduration=1.365818945 podStartE2EDuration="1.365818945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:26.36466054 +0000 UTC m=+260.857304404" watchObservedRunningTime="2026-07-18 02:01:26.365818945 +0000 UTC m=+260.858462773"
Jul 18 02:01:45.949671 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.949640    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"]
Jul 18 02:01:45.952005 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.951977    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:45.954720 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.954696    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 18 02:01:45.954804 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.954696    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 18 02:01:45.954804 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.954708    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 18 02:01:45.955815 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.955797    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 18 02:01:45.955857 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.955832    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 18 02:01:45.956006 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.955992    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wln6l\""
Jul 18 02:01:45.964583 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:45.964525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"]
Jul 18 02:01:46.116020 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.115997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88b9bb0d-897f-4b9d-88fb-9872e307d87e-metrics-cert\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.116164 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.116035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfvjl\" (UniqueName: \"kubernetes.io/projected/88b9bb0d-897f-4b9d-88fb-9872e307d87e-kube-api-access-xfvjl\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.116164 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.116091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88b9bb0d-897f-4b9d-88fb-9872e307d87e-cert\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.116164 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.116110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88b9bb0d-897f-4b9d-88fb-9872e307d87e-manager-config\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.217388 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.217325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfvjl\" (UniqueName: \"kubernetes.io/projected/88b9bb0d-897f-4b9d-88fb-9872e307d87e-kube-api-access-xfvjl\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.217388 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.217369    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88b9bb0d-897f-4b9d-88fb-9872e307d87e-cert\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.217388 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.217386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88b9bb0d-897f-4b9d-88fb-9872e307d87e-manager-config\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.217598 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.217428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88b9bb0d-897f-4b9d-88fb-9872e307d87e-metrics-cert\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.218047 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.218023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88b9bb0d-897f-4b9d-88fb-9872e307d87e-manager-config\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.219938 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.219908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88b9bb0d-897f-4b9d-88fb-9872e307d87e-cert\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.220015 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.219963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88b9bb0d-897f-4b9d-88fb-9872e307d87e-metrics-cert\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.231878 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.231856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfvjl\" (UniqueName: \"kubernetes.io/projected/88b9bb0d-897f-4b9d-88fb-9872e307d87e-kube-api-access-xfvjl\") pod \"lws-controller-manager-5b64b4b7d4-p6bqz\" (UID: \"88b9bb0d-897f-4b9d-88fb-9872e307d87e\") " pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.261847 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.261824    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:46.400818 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.400796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"]
Jul 18 02:01:46.403134 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:01:46.403103    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b9bb0d_897f_4b9d_88fb_9872e307d87e.slice/crio-7de590c492744d0a27c2472c6a87b7bc98e97bda5f6bc3ae2bc30d05220ea17b WatchSource:0}: Error finding container 7de590c492744d0a27c2472c6a87b7bc98e97bda5f6bc3ae2bc30d05220ea17b: Status 404 returned error can't find the container with id 7de590c492744d0a27c2472c6a87b7bc98e97bda5f6bc3ae2bc30d05220ea17b
Jul 18 02:01:46.426252 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:46.426222    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz" event={"ID":"88b9bb0d-897f-4b9d-88fb-9872e307d87e","Type":"ContainerStarted","Data":"7de590c492744d0a27c2472c6a87b7bc98e97bda5f6bc3ae2bc30d05220ea17b"}
Jul 18 02:01:49.446105 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:49.446066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz" event={"ID":"88b9bb0d-897f-4b9d-88fb-9872e307d87e","Type":"ContainerStarted","Data":"bd5ea57251ef69143f1fd4ff409794faba7a5b53d006a082e02dc16bd8a81330"}
Jul 18 02:01:49.471157 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:49.471111    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz" podStartSLOduration=2.024966639 podStartE2EDuration="4.471096758s" totalImagesPullingTime="2.446130119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:49.469935638 +0000 UTC m=+283.962579466" watchObservedRunningTime="2026-07-18 02:01:49.471096758 +0000 UTC m=+283.963740584"
Jul 18 02:01:50.450621 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.450590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:01:50.700076 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.700045    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"]
Jul 18 02:01:50.707022 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.706957    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.709830 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.709806    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 02:01:50.710261 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.710243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 02:01:50.710494 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.710476    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r5mvq\""
Jul 18 02:01:50.710770 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.710751    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 02:01:50.710981 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.710966    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 02:01:50.727178 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.727160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"]
Jul 18 02:01:50.856849 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.856823    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb54667-9934-49a0-8753-8c2d8f5abc15-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.856955 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.856870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb54667-9934-49a0-8753-8c2d8f5abc15-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.856955 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.856893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb54667-9934-49a0-8753-8c2d8f5abc15-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.856955 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.856943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb54667-9934-49a0-8753-8c2d8f5abc15-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.857054 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.856987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xtf6\" (UniqueName: \"kubernetes.io/projected/7cb54667-9934-49a0-8753-8c2d8f5abc15-kube-api-access-6xtf6\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.958277 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.958222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb54667-9934-49a0-8753-8c2d8f5abc15-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.958277 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.958256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb54667-9934-49a0-8753-8c2d8f5abc15-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.958277 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.958275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb54667-9934-49a0-8753-8c2d8f5abc15-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.958483 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.958298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xtf6\" (UniqueName: \"kubernetes.io/projected/7cb54667-9934-49a0-8753-8c2d8f5abc15-kube-api-access-6xtf6\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.958483 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.958342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb54667-9934-49a0-8753-8c2d8f5abc15-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.958646 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.958625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb54667-9934-49a0-8753-8c2d8f5abc15-manifests\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.960701 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.960679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb54667-9934-49a0-8753-8c2d8f5abc15-tmp\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.960804 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.960782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb54667-9934-49a0-8753-8c2d8f5abc15-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.960940 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.960924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb54667-9934-49a0-8753-8c2d8f5abc15-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:50.966781 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:50.966761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xtf6\" (UniqueName: \"kubernetes.io/projected/7cb54667-9934-49a0-8753-8c2d8f5abc15-kube-api-access-6xtf6\") pod \"opendatahub-operator-controller-manager-7d87dd68c9-rczg9\" (UID: \"7cb54667-9934-49a0-8753-8c2d8f5abc15\") " pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:51.017791 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:51.017766    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:51.149089 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:51.149063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"]
Jul 18 02:01:51.150723 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:01:51.150693    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb54667_9934_49a0_8753_8c2d8f5abc15.slice/crio-326a0126d97f1013be7d0251e0613beb2e7df79e3fe924a2c3d7869f36091e7c WatchSource:0}: Error finding container 326a0126d97f1013be7d0251e0613beb2e7df79e3fe924a2c3d7869f36091e7c: Status 404 returned error can't find the container with id 326a0126d97f1013be7d0251e0613beb2e7df79e3fe924a2c3d7869f36091e7c
Jul 18 02:01:51.457979 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:51.457935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9" event={"ID":"7cb54667-9934-49a0-8753-8c2d8f5abc15","Type":"ContainerStarted","Data":"326a0126d97f1013be7d0251e0613beb2e7df79e3fe924a2c3d7869f36091e7c"}
Jul 18 02:01:54.474933 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:54.474906    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9" event={"ID":"7cb54667-9934-49a0-8753-8c2d8f5abc15","Type":"ContainerStarted","Data":"a9abace855ecc9eb1a17d3e2d206d836b55d96bfe876defb3661bcae62bd0e8f"}
Jul 18 02:01:55.480539 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:55.480513    2576 generic.go:356] "Generic (PLEG): container finished" podID="7cb54667-9934-49a0-8753-8c2d8f5abc15" containerID="a9abace855ecc9eb1a17d3e2d206d836b55d96bfe876defb3661bcae62bd0e8f" exitCode=0
Jul 18 02:01:55.480844 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:55.480601    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9" event={"ID":"7cb54667-9934-49a0-8753-8c2d8f5abc15","Type":"ContainerDied","Data":"a9abace855ecc9eb1a17d3e2d206d836b55d96bfe876defb3661bcae62bd0e8f"}
Jul 18 02:01:56.486779 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:56.486744    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9" event={"ID":"7cb54667-9934-49a0-8753-8c2d8f5abc15","Type":"ContainerStarted","Data":"90ea663608f7f297368725ad51c5efb314515db1876467d30514497dabbc09d2"}
Jul 18 02:01:56.487174 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:56.486911    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:01:56.510422 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:01:56.510383    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9" podStartSLOduration=1.830819526 podStartE2EDuration="6.510370942s" totalImagesPullingTime="3.663009253s" totalInitContainerRuntime="1.016542163s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:01:56.508924348 +0000 UTC m=+291.001568186" watchObservedRunningTime="2026-07-18 02:01:56.510370942 +0000 UTC m=+291.003014767"
Jul 18 02:02:01.460433 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:01.460402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b64b4b7d4-p6bqz"
Jul 18 02:02:06.245535 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:06.245508    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 02:02:07.494020 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:07.493992    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d87dd68c9-rczg9"
Jul 18 02:02:21.129299 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.129264    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"]
Jul 18 02:02:21.134564 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.134545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.137215 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.137166    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 02:02:21.137349 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.137238    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 02:02:21.137349 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.137314    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 02:02:21.138500 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.138478    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 02:02:21.138622 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.138529    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7gsxn\""
Jul 18 02:02:21.144556 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.144534    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"]
Jul 18 02:02:21.225382 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.225357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tmp\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.225510 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.225398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd7hl\" (UniqueName: \"kubernetes.io/projected/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-kube-api-access-qd7hl\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.225510 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.225428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tls-certs\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.326243 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.326216    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tmp\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.326346 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.326268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd7hl\" (UniqueName: \"kubernetes.io/projected/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-kube-api-access-qd7hl\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.326346 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.326296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tls-certs\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.326454 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:02:21.326425    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 18 02:02:21.326506 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:02:21.326493    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tls-certs podName:3a1dee76-44c1-4f6b-b4dc-0e947ce72231 nodeName:}" failed. No retries permitted until 2026-07-18 02:02:21.826474259 +0000 UTC m=+316.319118080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tls-certs") pod "kube-auth-proxy-748465cbd5-grg8q" (UID: "3a1dee76-44c1-4f6b-b4dc-0e947ce72231") : secret "kube-auth-proxy-tls" not found
Jul 18 02:02:21.328386 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.328365    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tmp\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.341463 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.341444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd7hl\" (UniqueName: \"kubernetes.io/projected/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-kube-api-access-qd7hl\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.835768 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.835733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tls-certs\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:21.838120 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:21.838090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a1dee76-44c1-4f6b-b4dc-0e947ce72231-tls-certs\") pod \"kube-auth-proxy-748465cbd5-grg8q\" (UID: \"3a1dee76-44c1-4f6b-b4dc-0e947ce72231\") " pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:22.045973 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:22.045943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"
Jul 18 02:02:22.183473 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:22.183441    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-748465cbd5-grg8q"]
Jul 18 02:02:22.186828 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:02:22.186791    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1dee76_44c1_4f6b_b4dc_0e947ce72231.slice/crio-34145fd951c949d1be0b24b823754f9617b8d3c88a5154486d7e0e31cc621067 WatchSource:0}: Error finding container 34145fd951c949d1be0b24b823754f9617b8d3c88a5154486d7e0e31cc621067: Status 404 returned error can't find the container with id 34145fd951c949d1be0b24b823754f9617b8d3c88a5154486d7e0e31cc621067
Jul 18 02:02:22.189080 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:22.189066    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:02:22.611561 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:22.611528    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q" event={"ID":"3a1dee76-44c1-4f6b-b4dc-0e947ce72231","Type":"ContainerStarted","Data":"34145fd951c949d1be0b24b823754f9617b8d3c88a5154486d7e0e31cc621067"}
Jul 18 02:02:25.630673 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:25.630634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q" event={"ID":"3a1dee76-44c1-4f6b-b4dc-0e947ce72231","Type":"ContainerStarted","Data":"88e34db6e8a5ad8cf98fa2b9a3c4b1402bfad396f04dc6c8b230c8d97cd4f681"}
Jul 18 02:02:25.646522 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:02:25.646466    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-748465cbd5-grg8q" podStartSLOduration=2.282810782 podStartE2EDuration="4.646450421s" totalImagesPullingTime="2.363639639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:25.64599212 +0000 UTC m=+320.138635948" watchObservedRunningTime="2026-07-18 02:02:25.646450421 +0000 UTC m=+320.139094246"
Jul 18 02:04:02.578120 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.578088    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"]
Jul 18 02:04:02.581263 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.581242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.583874 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.583852    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 18 02:04:02.583987 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.583870    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 18 02:04:02.584128 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.584116    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p4c9m\""
Jul 18 02:04:02.585005 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.584987    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 02:04:02.585098 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.585038    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 02:04:02.590704 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.590682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"]
Jul 18 02:04:02.626103 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.626082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87qbr\" (UniqueName: \"kubernetes.io/projected/9f61267b-b93b-45b4-8332-e0a31416924c-kube-api-access-87qbr\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.626243 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.626144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9f61267b-b93b-45b4-8332-e0a31416924c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.626316 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.626261    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f61267b-b93b-45b4-8332-e0a31416924c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.727620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.727576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9f61267b-b93b-45b4-8332-e0a31416924c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.727723 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.727645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f61267b-b93b-45b4-8332-e0a31416924c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.727789 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:02.727755    2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 18 02:04:02.727789 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.727762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87qbr\" (UniqueName: \"kubernetes.io/projected/9f61267b-b93b-45b4-8332-e0a31416924c-kube-api-access-87qbr\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.727887 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:02.727820    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f61267b-b93b-45b4-8332-e0a31416924c-plugin-serving-cert podName:9f61267b-b93b-45b4-8332-e0a31416924c nodeName:}" failed. No retries permitted until 2026-07-18 02:04:03.227799958 +0000 UTC m=+417.720443770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9f61267b-b93b-45b4-8332-e0a31416924c-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-w8h5t" (UID: "9f61267b-b93b-45b4-8332-e0a31416924c") : secret "plugin-serving-cert" not found
Jul 18 02:04:02.728280 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.728264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9f61267b-b93b-45b4-8332-e0a31416924c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:02.739450 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:02.739420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87qbr\" (UniqueName: \"kubernetes.io/projected/9f61267b-b93b-45b4-8332-e0a31416924c-kube-api-access-87qbr\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:03.232682 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:03.232634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f61267b-b93b-45b4-8332-e0a31416924c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:03.234962 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:03.234943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f61267b-b93b-45b4-8332-e0a31416924c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w8h5t\" (UID: \"9f61267b-b93b-45b4-8332-e0a31416924c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:03.490855 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:03.490783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"
Jul 18 02:04:03.619948 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:03.619921    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t"]
Jul 18 02:04:03.621912 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:04:03.621882    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f61267b_b93b_45b4_8332_e0a31416924c.slice/crio-d46f6efb030ed4dbf8a64b74e7f8284f2828cb11c8f980c527ff50a58422b4e9 WatchSource:0}: Error finding container d46f6efb030ed4dbf8a64b74e7f8284f2828cb11c8f980c527ff50a58422b4e9: Status 404 returned error can't find the container with id d46f6efb030ed4dbf8a64b74e7f8284f2828cb11c8f980c527ff50a58422b4e9
Jul 18 02:04:04.086648 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:04.086618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t" event={"ID":"9f61267b-b93b-45b4-8332-e0a31416924c","Type":"ContainerStarted","Data":"d46f6efb030ed4dbf8a64b74e7f8284f2828cb11c8f980c527ff50a58422b4e9"}
Jul 18 02:04:11.862647 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:11.862610    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"]
Jul 18 02:04:11.872904 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:11.872874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:11.875501 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:11.875477    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8xdm8\""
Jul 18 02:04:11.886267 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:11.886241    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"]
Jul 18 02:04:11.911831 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:11.911801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzxx5\" (UniqueName: \"kubernetes.io/projected/12050669-adc8-4990-b1d3-593b9688fc0a-kube-api-access-fzxx5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:11.911949 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:11.911838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12050669-adc8-4990-b1d3-593b9688fc0a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:12.012813 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.012782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzxx5\" (UniqueName: \"kubernetes.io/projected/12050669-adc8-4990-b1d3-593b9688fc0a-kube-api-access-fzxx5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:12.012987 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.012828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12050669-adc8-4990-b1d3-593b9688fc0a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:12.013329 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.013277    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12050669-adc8-4990-b1d3-593b9688fc0a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:12.024212 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.024164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzxx5\" (UniqueName: \"kubernetes.io/projected/12050669-adc8-4990-b1d3-593b9688fc0a-kube-api-access-fzxx5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:12.185479 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.185405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:12.617773 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.617734    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"]
Jul 18 02:04:12.626951 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.626763    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"]
Jul 18 02:04:12.641167 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.641062    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"]
Jul 18 02:04:12.646733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.646710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.658382 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.658352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"]
Jul 18 02:04:12.720551 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.720508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sx58\" (UniqueName: \"kubernetes.io/projected/8a5d2c05-e624-4303-8451-136b9e93937f-kube-api-access-8sx58\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wlrtz\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.720737 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.720660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d2c05-e624-4303-8451-136b9e93937f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wlrtz\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.821684 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.821647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sx58\" (UniqueName: \"kubernetes.io/projected/8a5d2c05-e624-4303-8451-136b9e93937f-kube-api-access-8sx58\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wlrtz\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.821851 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.821740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d2c05-e624-4303-8451-136b9e93937f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wlrtz\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.822173 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.822141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d2c05-e624-4303-8451-136b9e93937f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wlrtz\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.833679 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.833654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sx58\" (UniqueName: \"kubernetes.io/projected/8a5d2c05-e624-4303-8451-136b9e93937f-kube-api-access-8sx58\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wlrtz\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:12.962366 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:12.962332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:25.029449 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:25.029413    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"]
Jul 18 02:04:25.030826 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:04:25.030796    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5d2c05_e624_4303_8451_136b9e93937f.slice/crio-4526699ffdb9b5cc6f349d39a0ef3fd5080350776df0ab65e37eb191aabbba1f WatchSource:0}: Error finding container 4526699ffdb9b5cc6f349d39a0ef3fd5080350776df0ab65e37eb191aabbba1f: Status 404 returned error can't find the container with id 4526699ffdb9b5cc6f349d39a0ef3fd5080350776df0ab65e37eb191aabbba1f
Jul 18 02:04:25.213153 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:25.213122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz" event={"ID":"8a5d2c05-e624-4303-8451-136b9e93937f","Type":"ContainerStarted","Data":"4526699ffdb9b5cc6f349d39a0ef3fd5080350776df0ab65e37eb191aabbba1f"}
Jul 18 02:04:26.219418 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:26.219376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t" event={"ID":"9f61267b-b93b-45b4-8332-e0a31416924c","Type":"ContainerStarted","Data":"5ecb0ef750a89c44ecbfb8e8af5c25d059ee2af20dc8192d3522f95c9257607e"}
Jul 18 02:04:26.236433 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:26.236384    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w8h5t" podStartSLOduration=2.588602706 podStartE2EDuration="24.236369599s" totalImagesPullingTime="21.647766893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:26.234652996 +0000 UTC m=+440.727296826" watchObservedRunningTime="2026-07-18 02:04:26.236369599 +0000 UTC m=+440.729013424"
Jul 18 02:04:27.445217 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:27.445152    2576 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-pdvdd_kuadrant-system_12050669-adc8-4990-b1d3-593b9688fc0a_0(0ce09c1798bb3c2da8bb1c78eabb1c5dad975f4a7773299544a7061662ef88ed): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-pdvdd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 18 02:04:27.445477 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:27.445262    2576 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-pdvdd_kuadrant-system_12050669-adc8-4990-b1d3-593b9688fc0a_0(0ce09c1798bb3c2da8bb1c78eabb1c5dad975f4a7773299544a7061662ef88ed): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-pdvdd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:28.229015 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.228940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:28.231257 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:28.231223    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" is forbidden: User \"system:node:ip-10-0-133-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-113.ec2.internal' and this object" podUID="12050669-adc8-4990-b1d3-593b9688fc0a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:28.236140 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.236116    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:28.238357 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:28.238327    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" is forbidden: User \"system:node:ip-10-0-133-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-113.ec2.internal' and this object" podUID="12050669-adc8-4990-b1d3-593b9688fc0a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:28.367608 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.367581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12050669-adc8-4990-b1d3-593b9688fc0a-extensions-socket-volume\") pod \"12050669-adc8-4990-b1d3-593b9688fc0a\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") "
Jul 18 02:04:28.367772 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.367664    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzxx5\" (UniqueName: \"kubernetes.io/projected/12050669-adc8-4990-b1d3-593b9688fc0a-kube-api-access-fzxx5\") pod \"12050669-adc8-4990-b1d3-593b9688fc0a\" (UID: \"12050669-adc8-4990-b1d3-593b9688fc0a\") "
Jul 18 02:04:28.367880 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.367856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12050669-adc8-4990-b1d3-593b9688fc0a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "12050669-adc8-4990-b1d3-593b9688fc0a" (UID: "12050669-adc8-4990-b1d3-593b9688fc0a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:04:28.369933 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.369906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12050669-adc8-4990-b1d3-593b9688fc0a-kube-api-access-fzxx5" (OuterVolumeSpecName: "kube-api-access-fzxx5") pod "12050669-adc8-4990-b1d3-593b9688fc0a" (UID: "12050669-adc8-4990-b1d3-593b9688fc0a"). InnerVolumeSpecName "kube-api-access-fzxx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:04:28.468265 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.468228    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12050669-adc8-4990-b1d3-593b9688fc0a-extensions-socket-volume\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:04:28.468265 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:28.468263    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzxx5\" (UniqueName: \"kubernetes.io/projected/12050669-adc8-4990-b1d3-593b9688fc0a-kube-api-access-fzxx5\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:04:29.088299 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:29.088261    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12050669-adc8-4990-b1d3-593b9688fc0a" path="/var/lib/kubelet/pods/12050669-adc8-4990-b1d3-593b9688fc0a/volumes"
Jul 18 02:04:29.234493 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:29.234462    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:29.237092 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:29.237049    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" is forbidden: User \"system:node:ip-10-0-133-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-113.ec2.internal' and this object" podUID="12050669-adc8-4990-b1d3-593b9688fc0a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:29.240223 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:29.240184    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" is forbidden: User \"system:node:ip-10-0-133-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-113.ec2.internal' and this object" podUID="12050669-adc8-4990-b1d3-593b9688fc0a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:30.241917 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:30.241827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz" event={"ID":"8a5d2c05-e624-4303-8451-136b9e93937f","Type":"ContainerStarted","Data":"c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d"}
Jul 18 02:04:30.242539 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:30.241964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:30.268474 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:30.268437    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-pdvdd\" is forbidden: User \"system:node:ip-10-0-133-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-113.ec2.internal' and this object" podUID="12050669-adc8-4990-b1d3-593b9688fc0a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-pdvdd"
Jul 18 02:04:30.268794 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:30.268755    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz" podStartSLOduration=13.540771665 podStartE2EDuration="18.268740462s" totalImagesPullingTime="4.727968797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:30.266112342 +0000 UTC m=+444.758756169" watchObservedRunningTime="2026-07-18 02:04:30.268740462 +0000 UTC m=+444.761384288"
Jul 18 02:04:41.248544 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:41.248510    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:53.238628 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.238586    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"]
Jul 18 02:04:53.239070 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.238807    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz" podUID="8a5d2c05-e624-4303-8451-136b9e93937f" containerName="manager" containerID="cri-o://c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d" gracePeriod=10
Jul 18 02:04:53.481364 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.481342    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:53.576638 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.576561    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d2c05-e624-4303-8451-136b9e93937f-extensions-socket-volume\") pod \"8a5d2c05-e624-4303-8451-136b9e93937f\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") "
Jul 18 02:04:53.576794 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.576644    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sx58\" (UniqueName: \"kubernetes.io/projected/8a5d2c05-e624-4303-8451-136b9e93937f-kube-api-access-8sx58\") pod \"8a5d2c05-e624-4303-8451-136b9e93937f\" (UID: \"8a5d2c05-e624-4303-8451-136b9e93937f\") "
Jul 18 02:04:53.577151 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.577115    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a5d2c05-e624-4303-8451-136b9e93937f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8a5d2c05-e624-4303-8451-136b9e93937f" (UID: "8a5d2c05-e624-4303-8451-136b9e93937f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:04:53.579044 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.579008    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5d2c05-e624-4303-8451-136b9e93937f-kube-api-access-8sx58" (OuterVolumeSpecName: "kube-api-access-8sx58") pod "8a5d2c05-e624-4303-8451-136b9e93937f" (UID: "8a5d2c05-e624-4303-8451-136b9e93937f"). InnerVolumeSpecName "kube-api-access-8sx58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:04:53.677681 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.677656    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sx58\" (UniqueName: \"kubernetes.io/projected/8a5d2c05-e624-4303-8451-136b9e93937f-kube-api-access-8sx58\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:04:53.677681 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:53.677678    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d2c05-e624-4303-8451-136b9e93937f-extensions-socket-volume\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:04:54.350491 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.350456    2576 generic.go:356] "Generic (PLEG): container finished" podID="8a5d2c05-e624-4303-8451-136b9e93937f" containerID="c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d" exitCode=0
Jul 18 02:04:54.350887 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.350511    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"
Jul 18 02:04:54.350887 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.350551    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz" event={"ID":"8a5d2c05-e624-4303-8451-136b9e93937f","Type":"ContainerDied","Data":"c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d"}
Jul 18 02:04:54.350887 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.350615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz" event={"ID":"8a5d2c05-e624-4303-8451-136b9e93937f","Type":"ContainerDied","Data":"4526699ffdb9b5cc6f349d39a0ef3fd5080350776df0ab65e37eb191aabbba1f"}
Jul 18 02:04:54.350887 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.350649    2576 scope.go:117] "RemoveContainer" containerID="c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d"
Jul 18 02:04:54.360921 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.360901    2576 scope.go:117] "RemoveContainer" containerID="c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d"
Jul 18 02:04:54.361224 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:04:54.361204    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d\": container with ID starting with c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d not found: ID does not exist" containerID="c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d"
Jul 18 02:04:54.361311 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.361235    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d"} err="failed to get container status \"c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d\": rpc error: code = NotFound desc = could not find container \"c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d\": container with ID starting with c26e50a44b1a1082856e56a51fff849d7c6c9ed25e21611cb714365f196c956d not found: ID does not exist"
Jul 18 02:04:54.373412 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.373391    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"]
Jul 18 02:04:54.379781 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:54.379763    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wlrtz"]
Jul 18 02:04:55.080376 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:04:55.080347    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5d2c05-e624-4303-8451-136b9e93937f" path="/var/lib/kubelet/pods/8a5d2c05-e624-4303-8451-136b9e93937f/volumes"
Jul 18 02:05:00.686938 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.686902    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:05:00.687429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.687281    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5d2c05-e624-4303-8451-136b9e93937f" containerName="manager"
Jul 18 02:05:00.687429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.687295    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5d2c05-e624-4303-8451-136b9e93937f" containerName="manager"
Jul 18 02:05:00.687429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.687386    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5d2c05-e624-4303-8451-136b9e93937f" containerName="manager"
Jul 18 02:05:00.689326 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.689306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.692108 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.692086    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 18 02:05:00.699868 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.699846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:05:00.715142 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.715123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:05:00.736973 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.736947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/59a3c995-844f-43c3-adc5-96c66692ecae-config-file\") pod \"limitador-limitador-54d6d988d5-s4hzq\" (UID: \"59a3c995-844f-43c3-adc5-96c66692ecae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.737088 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.736995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmdr8\" (UniqueName: \"kubernetes.io/projected/59a3c995-844f-43c3-adc5-96c66692ecae-kube-api-access-tmdr8\") pod \"limitador-limitador-54d6d988d5-s4hzq\" (UID: \"59a3c995-844f-43c3-adc5-96c66692ecae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.837535 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.837506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/59a3c995-844f-43c3-adc5-96c66692ecae-config-file\") pod \"limitador-limitador-54d6d988d5-s4hzq\" (UID: \"59a3c995-844f-43c3-adc5-96c66692ecae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.837619 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.837550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmdr8\" (UniqueName: \"kubernetes.io/projected/59a3c995-844f-43c3-adc5-96c66692ecae-kube-api-access-tmdr8\") pod \"limitador-limitador-54d6d988d5-s4hzq\" (UID: \"59a3c995-844f-43c3-adc5-96c66692ecae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.838163 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.838146    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/59a3c995-844f-43c3-adc5-96c66692ecae-config-file\") pod \"limitador-limitador-54d6d988d5-s4hzq\" (UID: \"59a3c995-844f-43c3-adc5-96c66692ecae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.846266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.846236    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmdr8\" (UniqueName: \"kubernetes.io/projected/59a3c995-844f-43c3-adc5-96c66692ecae-kube-api-access-tmdr8\") pod \"limitador-limitador-54d6d988d5-s4hzq\" (UID: \"59a3c995-844f-43c3-adc5-96c66692ecae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:00.999884 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:00.999861    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:01.132146 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.132123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:05:01.134810 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:05:01.134783    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a3c995_844f_43c3_adc5_96c66692ecae.slice/crio-b5f0b2265bc9ecd9b59709f22dcbcfed83f8dda2bbc7456c98db9dbe4312515e WatchSource:0}: Error finding container b5f0b2265bc9ecd9b59709f22dcbcfed83f8dda2bbc7456c98db9dbe4312515e: Status 404 returned error can't find the container with id b5f0b2265bc9ecd9b59709f22dcbcfed83f8dda2bbc7456c98db9dbe4312515e
Jul 18 02:05:01.351548 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.351485    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-zf6js"]
Jul 18 02:05:01.354608 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.354593    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:01.357598 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.357572    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hxsnd\""
Jul 18 02:05:01.363459 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.362901    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zf6js"]
Jul 18 02:05:01.384595 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.384571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq" event={"ID":"59a3c995-844f-43c3-adc5-96c66692ecae","Type":"ContainerStarted","Data":"b5f0b2265bc9ecd9b59709f22dcbcfed83f8dda2bbc7456c98db9dbe4312515e"}
Jul 18 02:05:01.443782 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.443756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb58j\" (UniqueName: \"kubernetes.io/projected/6a0816bd-0a59-487f-a5ad-64c86c4ec44a-kube-api-access-cb58j\") pod \"authorino-75d556bd77-zf6js\" (UID: \"6a0816bd-0a59-487f-a5ad-64c86c4ec44a\") " pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:01.545060 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.545033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb58j\" (UniqueName: \"kubernetes.io/projected/6a0816bd-0a59-487f-a5ad-64c86c4ec44a-kube-api-access-cb58j\") pod \"authorino-75d556bd77-zf6js\" (UID: \"6a0816bd-0a59-487f-a5ad-64c86c4ec44a\") " pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:01.552896 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.552865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb58j\" (UniqueName: \"kubernetes.io/projected/6a0816bd-0a59-487f-a5ad-64c86c4ec44a-kube-api-access-cb58j\") pod \"authorino-75d556bd77-zf6js\" (UID: \"6a0816bd-0a59-487f-a5ad-64c86c4ec44a\") " pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:01.670017 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.669955    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:01.831897 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:01.831680    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zf6js"]
Jul 18 02:05:01.835162 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:05:01.835131    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a0816bd_0a59_487f_a5ad_64c86c4ec44a.slice/crio-919763ccb70e859b81f75371765fcc2d403d27fa9e8659ee8d09f7f30b787258 WatchSource:0}: Error finding container 919763ccb70e859b81f75371765fcc2d403d27fa9e8659ee8d09f7f30b787258: Status 404 returned error can't find the container with id 919763ccb70e859b81f75371765fcc2d403d27fa9e8659ee8d09f7f30b787258
Jul 18 02:05:02.390310 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:02.390272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zf6js" event={"ID":"6a0816bd-0a59-487f-a5ad-64c86c4ec44a","Type":"ContainerStarted","Data":"919763ccb70e859b81f75371765fcc2d403d27fa9e8659ee8d09f7f30b787258"}
Jul 18 02:05:05.408755 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:05.408716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq" event={"ID":"59a3c995-844f-43c3-adc5-96c66692ecae","Type":"ContainerStarted","Data":"b3625523fa2057f7ef5efbc60df7802b0f27b0ab5632770ba50414c2ed33b777"}
Jul 18 02:05:05.409225 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:05.409086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:05.429332 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:05.429208    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq" podStartSLOduration=1.781278524 podStartE2EDuration="5.429173019s" totalImagesPullingTime="3.647894495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:05.427700428 +0000 UTC m=+479.920344255" watchObservedRunningTime="2026-07-18 02:05:05.429173019 +0000 UTC m=+479.921816844"
Jul 18 02:05:07.419658 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:07.419619    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zf6js" event={"ID":"6a0816bd-0a59-487f-a5ad-64c86c4ec44a","Type":"ContainerStarted","Data":"647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4"}
Jul 18 02:05:07.433348 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:07.433297    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-zf6js" podStartSLOduration=1.8192804 podStartE2EDuration="6.433279646s" totalImagesPullingTime="4.613999246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:05:07.432977484 +0000 UTC m=+481.925621311" watchObservedRunningTime="2026-07-18 02:05:07.433279646 +0000 UTC m=+481.925923472"
Jul 18 02:05:16.415264 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:16.415232    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"
Jul 18 02:05:37.108388 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.108356    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zf6js"]
Jul 18 02:05:37.108808 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.108525    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-zf6js" podUID="6a0816bd-0a59-487f-a5ad-64c86c4ec44a" containerName="authorino" containerID="cri-o://647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4" gracePeriod=30
Jul 18 02:05:37.367966 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.367911    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:37.449848 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.449819    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb58j\" (UniqueName: \"kubernetes.io/projected/6a0816bd-0a59-487f-a5ad-64c86c4ec44a-kube-api-access-cb58j\") pod \"6a0816bd-0a59-487f-a5ad-64c86c4ec44a\" (UID: \"6a0816bd-0a59-487f-a5ad-64c86c4ec44a\") "
Jul 18 02:05:37.451660 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.451635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0816bd-0a59-487f-a5ad-64c86c4ec44a-kube-api-access-cb58j" (OuterVolumeSpecName: "kube-api-access-cb58j") pod "6a0816bd-0a59-487f-a5ad-64c86c4ec44a" (UID: "6a0816bd-0a59-487f-a5ad-64c86c4ec44a"). InnerVolumeSpecName "kube-api-access-cb58j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:05:37.550708 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.550682    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb58j\" (UniqueName: \"kubernetes.io/projected/6a0816bd-0a59-487f-a5ad-64c86c4ec44a-kube-api-access-cb58j\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:05:37.554591 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.554567    2576 generic.go:356] "Generic (PLEG): container finished" podID="6a0816bd-0a59-487f-a5ad-64c86c4ec44a" containerID="647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4" exitCode=0
Jul 18 02:05:37.554705 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.554607    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zf6js"
Jul 18 02:05:37.554705 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.554617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zf6js" event={"ID":"6a0816bd-0a59-487f-a5ad-64c86c4ec44a","Type":"ContainerDied","Data":"647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4"}
Jul 18 02:05:37.554705 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.554648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zf6js" event={"ID":"6a0816bd-0a59-487f-a5ad-64c86c4ec44a","Type":"ContainerDied","Data":"919763ccb70e859b81f75371765fcc2d403d27fa9e8659ee8d09f7f30b787258"}
Jul 18 02:05:37.554705 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.554663    2576 scope.go:117] "RemoveContainer" containerID="647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4"
Jul 18 02:05:37.563374 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.563358    2576 scope.go:117] "RemoveContainer" containerID="647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4"
Jul 18 02:05:37.563630 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:05:37.563601    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4\": container with ID starting with 647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4 not found: ID does not exist" containerID="647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4"
Jul 18 02:05:37.563677 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.563627    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4"} err="failed to get container status \"647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4\": rpc error: code = NotFound desc = could not find container \"647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4\": container with ID starting with 647e65e3a5506a961746b70b0fe5ed77ab4f76fcacecafbaeb1fb00366a367a4 not found: ID does not exist"
Jul 18 02:05:37.575008 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.574986    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zf6js"]
Jul 18 02:05:37.578518 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:37.578499    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zf6js"]
Jul 18 02:05:39.080868 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:05:39.080839    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a0816bd-0a59-487f-a5ad-64c86c4ec44a" path="/var/lib/kubelet/pods/6a0816bd-0a59-487f-a5ad-64c86c4ec44a/volumes"
Jul 18 02:06:18.449354 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:18.449324    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:06:42.021793 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:42.021751    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:06:50.746487 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:50.746453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:06:56.025424 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.025389    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"]
Jul 18 02:06:56.025779 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.025758    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0816bd-0a59-487f-a5ad-64c86c4ec44a" containerName="authorino"
Jul 18 02:06:56.025779 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.025769    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0816bd-0a59-487f-a5ad-64c86c4ec44a" containerName="authorino"
Jul 18 02:06:56.025847 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.025820    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0816bd-0a59-487f-a5ad-64c86c4ec44a" containerName="authorino"
Jul 18 02:06:56.027826 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.027803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.032448 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.032421    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7gdmh\""
Jul 18 02:06:56.032566 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.032421    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 18 02:06:56.032566 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.032454    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 02:06:56.032732 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.032719    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 02:06:56.037063 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.037044    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"]
Jul 18 02:06:56.083641 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.083613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.083771 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.083645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2acfdadb-31e9-4fcd-93f7-2de584610d35-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.083771 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.083689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.083771 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.083737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.083889 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.083786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.083889 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.083817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkr4n\" (UniqueName: \"kubernetes.io/projected/2acfdadb-31e9-4fcd-93f7-2de584610d35-kube-api-access-nkr4n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.184784 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.184743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.184784 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.184782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2acfdadb-31e9-4fcd-93f7-2de584610d35-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185061 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.184834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185061 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.184851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185061 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.184879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185061 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.184979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkr4n\" (UniqueName: \"kubernetes.io/projected/2acfdadb-31e9-4fcd-93f7-2de584610d35-kube-api-access-nkr4n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185301 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.185253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185301 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.185268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.185410 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.185348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.187049 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.187020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2acfdadb-31e9-4fcd-93f7-2de584610d35-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.187236 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.187217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2acfdadb-31e9-4fcd-93f7-2de584610d35-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.193052 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.193029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkr4n\" (UniqueName: \"kubernetes.io/projected/2acfdadb-31e9-4fcd-93f7-2de584610d35-kube-api-access-nkr4n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8xvqb\" (UID: \"2acfdadb-31e9-4fcd-93f7-2de584610d35\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.338316 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.338222    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:06:56.465861 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.465836    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"]
Jul 18 02:06:56.467562 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:06:56.467531    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2acfdadb_31e9_4fcd_93f7_2de584610d35.slice/crio-bb4fa855361b6162933a85efdc76b427347334f4924406ef4af9eda697454e03 WatchSource:0}: Error finding container bb4fa855361b6162933a85efdc76b427347334f4924406ef4af9eda697454e03: Status 404 returned error can't find the container with id bb4fa855361b6162933a85efdc76b427347334f4924406ef4af9eda697454e03
Jul 18 02:06:56.945673 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:56.945633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb" event={"ID":"2acfdadb-31e9-4fcd-93f7-2de584610d35","Type":"ContainerStarted","Data":"bb4fa855361b6162933a85efdc76b427347334f4924406ef4af9eda697454e03"}
Jul 18 02:06:57.125481 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:57.125451    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:06:57.951848 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:57.951816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb" event={"ID":"2acfdadb-31e9-4fcd-93f7-2de584610d35","Type":"ContainerStarted","Data":"58ca1325edb61ba2182aefae67df48a6d305b34d4c01c87bebab05808e959e01"}
Jul 18 02:06:57.972645 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:06:57.972597    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb" podStartSLOduration=0.988609398 podStartE2EDuration="1.972579546s" totalImagesPullingTime="983.970148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:06:57.971363949 +0000 UTC m=+592.464007776" watchObservedRunningTime="2026-07-18 02:06:57.972579546 +0000 UTC m=+592.465223373"
Jul 18 02:07:06.338698 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:06.338664    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:07:06.339165 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:06.338770    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:07:06.350894 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:06.350872    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:07:07.004173 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:07.004142    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8xvqb"
Jul 18 02:07:26.141653 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.141620    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"]
Jul 18 02:07:26.143840 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.143825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.146611 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.146589    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 18 02:07:26.155473 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.155447    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"]
Jul 18 02:07:26.229187 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.229163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.229318 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.229213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.229318 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.229262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r5zr\" (UniqueName: \"kubernetes.io/projected/766eb2a3-9814-4993-81d7-c011d0824530-kube-api-access-7r5zr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.229318 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.229280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.229480 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.229360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/766eb2a3-9814-4993-81d7-c011d0824530-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.229480 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.229430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.330815 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.330786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r5zr\" (UniqueName: \"kubernetes.io/projected/766eb2a3-9814-4993-81d7-c011d0824530-kube-api-access-7r5zr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.330917 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.330822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.330917 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.330881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/766eb2a3-9814-4993-81d7-c011d0824530-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.331028 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.330923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.331114 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.331095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.331171 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.331125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.331284 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.331264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.331342 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.331296    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.331417 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.331402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.333185 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.333166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/766eb2a3-9814-4993-81d7-c011d0824530-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.333444 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.333428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/766eb2a3-9814-4993-81d7-c011d0824530-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.338631 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.338609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r5zr\" (UniqueName: \"kubernetes.io/projected/766eb2a3-9814-4993-81d7-c011d0824530-kube-api-access-7r5zr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2\" (UID: \"766eb2a3-9814-4993-81d7-c011d0824530\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.454495 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.454472    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:26.584547 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.584519    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"]
Jul 18 02:07:26.586539 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:07:26.586509    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766eb2a3_9814_4993_81d7_c011d0824530.slice/crio-2ffc14dec14b89a34074559204e04877a37a822635171bd3f2b64f6f9585d9b8 WatchSource:0}: Error finding container 2ffc14dec14b89a34074559204e04877a37a822635171bd3f2b64f6f9585d9b8: Status 404 returned error can't find the container with id 2ffc14dec14b89a34074559204e04877a37a822635171bd3f2b64f6f9585d9b8
Jul 18 02:07:26.587812 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:26.587795    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:07:27.086076 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:27.085987    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2" event={"ID":"766eb2a3-9814-4993-81d7-c011d0824530","Type":"ContainerStarted","Data":"a55d2f775a88b90dc046d93bfd1f8cb7526313296227a0bf99e1be1d2a42377b"}
Jul 18 02:07:27.086076 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:27.086047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2" event={"ID":"766eb2a3-9814-4993-81d7-c011d0824530","Type":"ContainerStarted","Data":"2ffc14dec14b89a34074559204e04877a37a822635171bd3f2b64f6f9585d9b8"}
Jul 18 02:07:27.106585 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:27.106535    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2" podStartSLOduration=0.932587834 podStartE2EDuration="1.106521326s" totalImagesPullingTime="173.933492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:07:27.104914662 +0000 UTC m=+621.597558488" watchObservedRunningTime="2026-07-18 02:07:27.106521326 +0000 UTC m=+621.599165129"
Jul 18 02:07:27.923226 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:27.923179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:07:36.455354 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:36.455317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:36.455725 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:36.455386    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:36.468073 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:36.468051    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:37.140294 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:37.140263    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2"
Jul 18 02:07:50.729312 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:50.729278    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:07:52.328835 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.328804    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"]
Jul 18 02:07:52.340551 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.340524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.343356 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.343328    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 18 02:07:52.344813 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.344787    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"]
Jul 18 02:07:52.439289 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.439261    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.439289 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.439290    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxc4b\" (UniqueName: \"kubernetes.io/projected/dce77d3b-3729-4522-93c3-b233fab7c4ea-kube-api-access-vxc4b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.439435 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.439363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.439435 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.439398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.439435 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.439415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.439435 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.439434    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dce77d3b-3729-4522-93c3-b233fab7c4ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.540712 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.540670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.540833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.540723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.540833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.540751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.540833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.540779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dce77d3b-3729-4522-93c3-b233fab7c4ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.540952 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.540858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.540952 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.540885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxc4b\" (UniqueName: \"kubernetes.io/projected/dce77d3b-3729-4522-93c3-b233fab7c4ea-kube-api-access-vxc4b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.541088 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.541064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.541213 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.541175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.541292 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.541274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.542972 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.542950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dce77d3b-3729-4522-93c3-b233fab7c4ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.543271 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.543253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dce77d3b-3729-4522-93c3-b233fab7c4ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.549451 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.549431    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxc4b\" (UniqueName: \"kubernetes.io/projected/dce77d3b-3729-4522-93c3-b233fab7c4ea-kube-api-access-vxc4b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf\" (UID: \"dce77d3b-3729-4522-93c3-b233fab7c4ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.651984 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.651913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:07:52.779785 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:52.779759    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"]
Jul 18 02:07:52.782095 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:07:52.782072    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddce77d3b_3729_4522_93c3_b233fab7c4ea.slice/crio-f936e0ba7ce69974e26a25373883e5bbd9adefa105136a7f1ca62c636ebb1363 WatchSource:0}: Error finding container f936e0ba7ce69974e26a25373883e5bbd9adefa105136a7f1ca62c636ebb1363: Status 404 returned error can't find the container with id f936e0ba7ce69974e26a25373883e5bbd9adefa105136a7f1ca62c636ebb1363
Jul 18 02:07:53.201396 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:53.201361    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf" event={"ID":"dce77d3b-3729-4522-93c3-b233fab7c4ea","Type":"ContainerStarted","Data":"fde023a92582667afa8ce20466ae3e518770b7e16db60ea706521be2be6fb010"}
Jul 18 02:07:53.201396 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:53.201400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf" event={"ID":"dce77d3b-3729-4522-93c3-b233fab7c4ea","Type":"ContainerStarted","Data":"f936e0ba7ce69974e26a25373883e5bbd9adefa105136a7f1ca62c636ebb1363"}
Jul 18 02:07:53.219327 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:07:53.219243    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf" podStartSLOduration=1.046805253 podStartE2EDuration="1.21923046s" totalImagesPullingTime="172.425207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:07:53.217116358 +0000 UTC m=+647.709760198" watchObservedRunningTime="2026-07-18 02:07:53.21923046 +0000 UTC m=+647.711874286"
Jul 18 02:08:01.736028 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:08:01.735988    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:08:02.652071 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:08:02.652036    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:08:02.652282 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:08:02.652116    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:08:02.664253 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:08:02.664229    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:08:03.258427 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:08:03.258402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf"
Jul 18 02:09:52.328789 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:09:52.328754    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:10:04.128881 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:10:04.128846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:10:13.829266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:10:13.829236    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:11:56.924642 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:11:56.924595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:12:06.138775 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:12:06.138700    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:12:18.935226 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:12:18.935178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:12:35.327679 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:12:35.327644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:12:45.530537 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:12:45.530502    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:12:58.122735 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:12:58.122694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:13:09.241258 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:13:09.241225    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:13:18.030054 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:13:18.030017    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:13:28.034244 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:13:28.034140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:13:40.125834 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:13:40.125799    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:13:50.130213 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:13:50.130161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:14:11.925905 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:14:11.925871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:14:24.625444 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:14:24.625412    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:14:46.671499 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:14:46.671465    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:14:59.234764 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:14:59.234692    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:15:15.227482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:15:15.227446    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:15:24.520143 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:15:24.520106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:15:34.824469 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:15:34.824423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:15:43.933011 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:15:43.932975    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:15:54.935327 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:15:54.935290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:17:41.928987 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:17:41.928953    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:17:54.230066 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:17:54.230030    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:18:03.027130 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:18:03.027046    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:18:13.864906 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:18:13.864868    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:18:23.929668 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:18:23.929632    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:18:34.326072 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:18:34.326034    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:18:44.629039 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:18:44.628998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:18:54.018770 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:18:54.018740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:19:06.325571 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:19:06.325537    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:19:15.126044 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:19:15.126009    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:19:26.525999 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:19:26.525964    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:19:47.084115 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:19:47.084037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:19:56.429437 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:19:56.429401    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:20:05.432429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:20:05.432394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:20:19.329926 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:20:19.329879    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:20:31.230695 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:20:31.230653    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:20:40.925632 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:20:40.925601    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:20:50.432575 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:20:50.432540    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:20:59.933428 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:20:59.933348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:21:09.638277 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:21:09.638238    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:21:19.028619 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:21:19.028584    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:21:32.329201 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:21:32.329151    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:21:41.230986 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:21:41.230949    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:21:51.230913 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:21:51.230882    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:22:01.727576 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:22:01.727537    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:22:14.732842 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:22:14.732802    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:22:24.425678 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:22:24.425646    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:22:36.028941 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:22:36.028903    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:22:48.650166 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:22:48.650128    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:23:03.036140 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:23:03.036102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:23:14.946444 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:23:14.946410    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:23:26.338293 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:23:26.338254    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:23:38.331729 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:23:38.331691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:23:48.533005 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:23:48.532967    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:24:17.337002 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:24:17.336921    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:24:28.839750 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:24:28.839715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:25:00.576230 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:25:00.576179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:25:10.735770 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:25:10.735730    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:25:45.835604 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:25:45.835512    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:25:57.135634 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:25:57.135598    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:26:29.931437 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:26:29.931393    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:27:59.435981 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:27:59.435941    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:28:09.732275 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:28:09.732232    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:28:21.925263 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:28:21.925226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:28:41.531619 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:28:41.531539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:29:19.528733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:19.528692    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:29:34.331972 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.331942    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"]
Jul 18 02:29:34.335413 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.335387    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.338305 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.338280    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-327441da\"/\"default-dockercfg-fz8th\""
Jul 18 02:29:34.339892 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.339870    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-327441da\"/\"kube-root-ca.crt\""
Jul 18 02:29:34.340086 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.340067    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-327441da\"/\"e2e-lifecycle-model-327441da-kserve-self-signed-certs\""
Jul 18 02:29:34.340175 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.339998    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-327441da\"/\"openshift-service-ca.crt\""
Jul 18 02:29:34.348728 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.348707    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"]
Jul 18 02:29:34.415000 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.414974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-dshm\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.415128 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.415008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ef585-20fd-4b51-807c-29c0f26b2284-tls-certs\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.415128 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.415034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-home\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.415240 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.415172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-model-cache\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.415240 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.415219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-tmp-dir\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.415310 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.415288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m9q6\" (UniqueName: \"kubernetes.io/projected/2c6ef585-20fd-4b51-807c-29c0f26b2284-kube-api-access-4m9q6\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.515833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.515807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-home\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.515944 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.515859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-model-cache\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.515944 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.515876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-tmp-dir\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.515944 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.515902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m9q6\" (UniqueName: \"kubernetes.io/projected/2c6ef585-20fd-4b51-807c-29c0f26b2284-kube-api-access-4m9q6\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.515944 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.515939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-dshm\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.516147 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.515957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ef585-20fd-4b51-807c-29c0f26b2284-tls-certs\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.516274 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.516254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-home\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.516274 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.516268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-model-cache\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.516390 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.516334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-tmp-dir\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.518144 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.518119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-dshm\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.518246 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.518223    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ef585-20fd-4b51-807c-29c0f26b2284-tls-certs\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.523824 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.523806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m9q6\" (UniqueName: \"kubernetes.io/projected/2c6ef585-20fd-4b51-807c-29c0f26b2284-kube-api-access-4m9q6\") pod \"e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") " pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.645369 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.645319    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:34.777338 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:34.777302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"]
Jul 18 02:29:34.780229 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:29:34.780169    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c6ef585_20fd_4b51_807c_29c0f26b2284.slice/crio-34049cf82c9cd0cd1f109be1d3a55347e4c60e051cd51b32262a16ae24da0f70 WatchSource:0}: Error finding container 34049cf82c9cd0cd1f109be1d3a55347e4c60e051cd51b32262a16ae24da0f70: Status 404 returned error can't find the container with id 34049cf82c9cd0cd1f109be1d3a55347e4c60e051cd51b32262a16ae24da0f70
Jul 18 02:29:35.298449 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:35.298411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2" event={"ID":"2c6ef585-20fd-4b51-807c-29c0f26b2284","Type":"ContainerStarted","Data":"8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb"}
Jul 18 02:29:35.298625 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:35.298460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2" event={"ID":"2c6ef585-20fd-4b51-807c-29c0f26b2284","Type":"ContainerStarted","Data":"34049cf82c9cd0cd1f109be1d3a55347e4c60e051cd51b32262a16ae24da0f70"}
Jul 18 02:29:35.335427 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:35.335373    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2" podStartSLOduration=1.3353596890000001 podStartE2EDuration="1.335359689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:29:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:29:35.334009095 +0000 UTC m=+1949.826652923" watchObservedRunningTime="2026-07-18 02:29:35.335359689 +0000 UTC m=+1949.828003514"
Jul 18 02:29:37.536581 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:37.536548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:29:44.646289 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:44.646239    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:44.646706 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:44.646317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:44.658736 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:44.658711    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:45.354069 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:45.354039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:29:52.843099 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:29:52.843064    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:30:10.437152 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:10.437071    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:30:19.440881 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:19.440843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:30:23.669993 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:23.669955    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"]
Jul 18 02:30:38.690716 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.690659    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2" podUID="2c6ef585-20fd-4b51-807c-29c0f26b2284" containerName="main" containerID="cri-o://8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb" gracePeriod=45
Jul 18 02:30:38.943558 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.943532    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:30:38.956401 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956367    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m9q6\" (UniqueName: \"kubernetes.io/projected/2c6ef585-20fd-4b51-807c-29c0f26b2284-kube-api-access-4m9q6\") pod \"2c6ef585-20fd-4b51-807c-29c0f26b2284\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") "
Jul 18 02:30:38.956535 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956429    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-tmp-dir\") pod \"2c6ef585-20fd-4b51-807c-29c0f26b2284\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") "
Jul 18 02:30:38.956535 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956451    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-dshm\") pod \"2c6ef585-20fd-4b51-807c-29c0f26b2284\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") "
Jul 18 02:30:38.956647 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956616    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-home\") pod \"2c6ef585-20fd-4b51-807c-29c0f26b2284\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") "
Jul 18 02:30:38.956700 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956646    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-model-cache\") pod \"2c6ef585-20fd-4b51-807c-29c0f26b2284\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") "
Jul 18 02:30:38.956700 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956690    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ef585-20fd-4b51-807c-29c0f26b2284-tls-certs\") pod \"2c6ef585-20fd-4b51-807c-29c0f26b2284\" (UID: \"2c6ef585-20fd-4b51-807c-29c0f26b2284\") "
Jul 18 02:30:38.956790 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.956698    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c6ef585-20fd-4b51-807c-29c0f26b2284" (UID: "2c6ef585-20fd-4b51-807c-29c0f26b2284"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:38.957186 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.957160    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-home" (OuterVolumeSpecName: "home") pod "2c6ef585-20fd-4b51-807c-29c0f26b2284" (UID: "2c6ef585-20fd-4b51-807c-29c0f26b2284"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:38.957530 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.957480    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c6ef585-20fd-4b51-807c-29c0f26b2284" (UID: "2c6ef585-20fd-4b51-807c-29c0f26b2284"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:38.960139 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.957814    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-home\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:30:38.960139 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.957844    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-model-cache\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:30:38.960139 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.957862    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-tmp-dir\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:30:38.960139 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.959911    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c6ef585-20fd-4b51-807c-29c0f26b2284-kube-api-access-4m9q6" (OuterVolumeSpecName: "kube-api-access-4m9q6") pod "2c6ef585-20fd-4b51-807c-29c0f26b2284" (UID: "2c6ef585-20fd-4b51-807c-29c0f26b2284"). InnerVolumeSpecName "kube-api-access-4m9q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:38.961356 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.961322    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c6ef585-20fd-4b51-807c-29c0f26b2284-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c6ef585-20fd-4b51-807c-29c0f26b2284" (UID: "2c6ef585-20fd-4b51-807c-29c0f26b2284"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:38.965155 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:38.965129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-dshm" (OuterVolumeSpecName: "dshm") pod "2c6ef585-20fd-4b51-807c-29c0f26b2284" (UID: "2c6ef585-20fd-4b51-807c-29c0f26b2284"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:39.058824 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.058797    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ef585-20fd-4b51-807c-29c0f26b2284-tls-certs\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:30:39.058824 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.058820    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m9q6\" (UniqueName: \"kubernetes.io/projected/2c6ef585-20fd-4b51-807c-29c0f26b2284-kube-api-access-4m9q6\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:30:39.058824 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.058829    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ef585-20fd-4b51-807c-29c0f26b2284-dshm\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:30:39.595657 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.595619    2576 generic.go:356] "Generic (PLEG): container finished" podID="2c6ef585-20fd-4b51-807c-29c0f26b2284" containerID="8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb" exitCode=0
Jul 18 02:30:39.595823 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.595684    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"
Jul 18 02:30:39.595823 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.595709    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2" event={"ID":"2c6ef585-20fd-4b51-807c-29c0f26b2284","Type":"ContainerDied","Data":"8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb"}
Jul 18 02:30:39.595823 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.595758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2" event={"ID":"2c6ef585-20fd-4b51-807c-29c0f26b2284","Type":"ContainerDied","Data":"34049cf82c9cd0cd1f109be1d3a55347e4c60e051cd51b32262a16ae24da0f70"}
Jul 18 02:30:39.595823 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.595777    2576 scope.go:117] "RemoveContainer" containerID="8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb"
Jul 18 02:30:39.604278 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.604257    2576 scope.go:117] "RemoveContainer" containerID="8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb"
Jul 18 02:30:39.604552 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:30:39.604533    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb\": container with ID starting with 8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb not found: ID does not exist" containerID="8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb"
Jul 18 02:30:39.604619 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.604565    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb"} err="failed to get container status \"8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb\": rpc error: code = NotFound desc = could not find container \"8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb\": container with ID starting with 8c0849d30d54a25bff9dde989a88feffcb26eb7cfa829fc133f264cb596cdbeb not found: ID does not exist"
Jul 18 02:30:39.612697 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.612674    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"]
Jul 18 02:30:39.616707 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:39.616687    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-327441da/e2e-lifecycle-model-327441da-kserve-7bdf4dccd7-5ksx2"]
Jul 18 02:30:41.081899 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:30:41.081866    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c6ef585-20fd-4b51-807c-29c0f26b2284" path="/var/lib/kubelet/pods/2c6ef585-20fd-4b51-807c-29c0f26b2284/volumes"
Jul 18 02:31:24.830675 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:31:24.830641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:31:34.633246 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:31:34.633168    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:32:25.618099 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:25.618058    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:32:42.626662 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:42.626629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:32:49.130842 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.130813    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"]
Jul 18 02:32:49.131294 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.131182    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c6ef585-20fd-4b51-807c-29c0f26b2284" containerName="main"
Jul 18 02:32:49.131294 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.131211    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c6ef585-20fd-4b51-807c-29c0f26b2284" containerName="main"
Jul 18 02:32:49.131294 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.131266    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c6ef585-20fd-4b51-807c-29c0f26b2284" containerName="main"
Jul 18 02:32:49.134184 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.134169    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.136710 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.136679    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9e2f67\"/\"openshift-service-ca.crt\""
Jul 18 02:32:49.137777 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.137758    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9e2f67\"/\"default-dockercfg-xrx45\""
Jul 18 02:32:49.137777 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.137778    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9e2f67\"/\"kube-root-ca.crt\""
Jul 18 02:32:49.137955 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.137829    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9e2f67\"/\"test-model-9e2f67-kserve-self-signed-certs\""
Jul 18 02:32:49.141918 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.141894    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"]
Jul 18 02:32:49.255437 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.255406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-dshm\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.255573 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.255455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-home\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.255573 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.255505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tmp-dir\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.255653 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.255572    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-model-cache\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.255653 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.255594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tls-certs\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.255653 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.255609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs585\" (UniqueName: \"kubernetes.io/projected/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-kube-api-access-gs585\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.356652 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.356627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tmp-dir\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.356779 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.356680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-model-cache\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.356840 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.356790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tls-certs\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.356840 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.356817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs585\" (UniqueName: \"kubernetes.io/projected/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-kube-api-access-gs585\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.356931 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.356852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-dshm\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.356931 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.356903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-home\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.357111 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.357086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-model-cache\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.357170 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.357152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tmp-dir\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.357327 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.357304    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-home\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.359164 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.359142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-dshm\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.359445 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.359425    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tls-certs\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.364915 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.364896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs585\" (UniqueName: \"kubernetes.io/projected/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-kube-api-access-gs585\") pod \"test-model-9e2f67-kserve-6f46c958dc-8qs8w\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") " pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.443930 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.443910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:49.572950 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:49.572922    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"]
Jul 18 02:32:49.574900 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:32:49.574873    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda29c8cc4_cdbe_49b0_9f42_c7cb70708721.slice/crio-8f0b3cfde8185a98735784697102e30f4d369f637e2cd1168fe5b46b1cdfee91 WatchSource:0}: Error finding container 8f0b3cfde8185a98735784697102e30f4d369f637e2cd1168fe5b46b1cdfee91: Status 404 returned error can't find the container with id 8f0b3cfde8185a98735784697102e30f4d369f637e2cd1168fe5b46b1cdfee91
Jul 18 02:32:50.204899 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:50.204867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w" event={"ID":"a29c8cc4-cdbe-49b0-9f42-c7cb70708721","Type":"ContainerStarted","Data":"f2f6f79bccdc54cb5b28ca7900df92e782d1f6dc3e507254a3fdc4b722e5416e"}
Jul 18 02:32:50.205277 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:50.204904    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w" event={"ID":"a29c8cc4-cdbe-49b0-9f42-c7cb70708721","Type":"ContainerStarted","Data":"8f0b3cfde8185a98735784697102e30f4d369f637e2cd1168fe5b46b1cdfee91"}
Jul 18 02:32:50.226081 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:50.226033    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w" podStartSLOduration=1.226018172 podStartE2EDuration="1.226018172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:50.224148883 +0000 UTC m=+2144.716792710" watchObservedRunningTime="2026-07-18 02:32:50.226018172 +0000 UTC m=+2144.718661997"
Jul 18 02:32:53.839789 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:53.839753    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:32:59.444642 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:59.444617    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:59.447810 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:59.444664    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:32:59.456989 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:32:59.456966    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:33:00.268794 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:33:00.268763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:33:42.655326 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:33:42.655294    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:33:56.835975 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:33:56.835935    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:34:03.445956 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:03.445918    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"]
Jul 18 02:34:04.564350 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.564323    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"]
Jul 18 02:34:04.567948 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.567930    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:04.570627 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.570607    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:34:04.570845 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.570831    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:34:04.575780 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.575756    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"]
Jul 18 02:34:04.686150 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.686125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90354506-3ccf-43ff-a26b-5d973e774e61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr\" (UID: \"90354506-3ccf-43ff-a26b-5d973e774e61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:04.786960 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.786930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90354506-3ccf-43ff-a26b-5d973e774e61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr\" (UID: \"90354506-3ccf-43ff-a26b-5d973e774e61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:04.787524 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.787506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90354506-3ccf-43ff-a26b-5d973e774e61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr\" (UID: \"90354506-3ccf-43ff-a26b-5d973e774e61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:04.878949 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:04.878891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:05.005800 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:05.005776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"]
Jul 18 02:34:05.008028 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:34:05.008005    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90354506_3ccf_43ff_a26b_5d973e774e61.slice/crio-b7c5289deaa3648c1309a54424a40a21803a45319ec841df2e9443106edd131b WatchSource:0}: Error finding container b7c5289deaa3648c1309a54424a40a21803a45319ec841df2e9443106edd131b: Status 404 returned error can't find the container with id b7c5289deaa3648c1309a54424a40a21803a45319ec841df2e9443106edd131b
Jul 18 02:34:05.010097 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:05.009915    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:34:05.563324 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:05.563292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr" event={"ID":"90354506-3ccf-43ff-a26b-5d973e774e61","Type":"ContainerStarted","Data":"b7c5289deaa3648c1309a54424a40a21803a45319ec841df2e9443106edd131b"}
Jul 18 02:34:07.575482 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:07.575449    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr" event={"ID":"90354506-3ccf-43ff-a26b-5d973e774e61","Type":"ContainerStarted","Data":"af36ed4f14c329687bb0a1e674f7739039dcae2799bb9de3feaff10b8c406ed5"}
Jul 18 02:34:07.594224 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:07.594134    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr" podStartSLOduration=1.527775517 podStartE2EDuration="3.594119102s" totalImagesPullingTime="2.066343585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:07.592115673 +0000 UTC m=+2222.084759499" watchObservedRunningTime="2026-07-18 02:34:07.594119102 +0000 UTC m=+2222.086762929"
Jul 18 02:34:08.582076 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:08.582044    2576 generic.go:356] "Generic (PLEG): container finished" podID="90354506-3ccf-43ff-a26b-5d973e774e61" containerID="af36ed4f14c329687bb0a1e674f7739039dcae2799bb9de3feaff10b8c406ed5" exitCode=0
Jul 18 02:34:08.582463 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:08.582087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr" event={"ID":"90354506-3ccf-43ff-a26b-5d973e774e61","Type":"ContainerDied","Data":"af36ed4f14c329687bb0a1e674f7739039dcae2799bb9de3feaff10b8c406ed5"}
Jul 18 02:34:09.716252 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:09.716229    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:09.728565 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:09.728547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90354506-3ccf-43ff-a26b-5d973e774e61-maas-api-service-ca\") pod \"90354506-3ccf-43ff-a26b-5d973e774e61\" (UID: \"90354506-3ccf-43ff-a26b-5d973e774e61\") "
Jul 18 02:34:09.728912 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:09.728890    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90354506-3ccf-43ff-a26b-5d973e774e61-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90354506-3ccf-43ff-a26b-5d973e774e61" (UID: "90354506-3ccf-43ff-a26b-5d973e774e61"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:34:09.829826 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:09.829801    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90354506-3ccf-43ff-a26b-5d973e774e61-maas-api-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:10.346347 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:10.346312    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:34:10.592934 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:10.592907    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"
Jul 18 02:34:10.593104 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:10.592899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr" event={"ID":"90354506-3ccf-43ff-a26b-5d973e774e61","Type":"ContainerDied","Data":"b7c5289deaa3648c1309a54424a40a21803a45319ec841df2e9443106edd131b"}
Jul 18 02:34:10.593104 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:10.593035    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7c5289deaa3648c1309a54424a40a21803a45319ec841df2e9443106edd131b"
Jul 18 02:34:18.466302 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.466253    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w" podUID="a29c8cc4-cdbe-49b0-9f42-c7cb70708721" containerName="main" containerID="cri-o://f2f6f79bccdc54cb5b28ca7900df92e782d1f6dc3e507254a3fdc4b722e5416e" gracePeriod=45
Jul 18 02:34:18.631955 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.631922    2576 generic.go:356] "Generic (PLEG): container finished" podID="a29c8cc4-cdbe-49b0-9f42-c7cb70708721" containerID="f2f6f79bccdc54cb5b28ca7900df92e782d1f6dc3e507254a3fdc4b722e5416e" exitCode=0
Jul 18 02:34:18.632094 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.632000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w" event={"ID":"a29c8cc4-cdbe-49b0-9f42-c7cb70708721","Type":"ContainerDied","Data":"f2f6f79bccdc54cb5b28ca7900df92e782d1f6dc3e507254a3fdc4b722e5416e"}
Jul 18 02:34:18.705267 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.705228    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:34:18.801143 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801070    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tmp-dir\") pod \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") "
Jul 18 02:34:18.801143 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801114    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-model-cache\") pod \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") "
Jul 18 02:34:18.801351 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801146    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tls-certs\") pod \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") "
Jul 18 02:34:18.801351 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801221    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs585\" (UniqueName: \"kubernetes.io/projected/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-kube-api-access-gs585\") pod \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") "
Jul 18 02:34:18.801437 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801341    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-model-cache" (OuterVolumeSpecName: "model-cache") pod "a29c8cc4-cdbe-49b0-9f42-c7cb70708721" (UID: "a29c8cc4-cdbe-49b0-9f42-c7cb70708721"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:18.801437 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801406    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-dshm\") pod \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") "
Jul 18 02:34:18.801521 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801447    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-home\") pod \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\" (UID: \"a29c8cc4-cdbe-49b0-9f42-c7cb70708721\") "
Jul 18 02:34:18.801521 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a29c8cc4-cdbe-49b0-9f42-c7cb70708721" (UID: "a29c8cc4-cdbe-49b0-9f42-c7cb70708721"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:18.801735 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801708    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-home" (OuterVolumeSpecName: "home") pod "a29c8cc4-cdbe-49b0-9f42-c7cb70708721" (UID: "a29c8cc4-cdbe-49b0-9f42-c7cb70708721"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:18.801833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801781    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tmp-dir\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:18.801833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801792    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-model-cache\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:18.801833 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.801802    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-home\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:18.803465 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.803435    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a29c8cc4-cdbe-49b0-9f42-c7cb70708721" (UID: "a29c8cc4-cdbe-49b0-9f42-c7cb70708721"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:34:18.803465 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.803440    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-dshm" (OuterVolumeSpecName: "dshm") pod "a29c8cc4-cdbe-49b0-9f42-c7cb70708721" (UID: "a29c8cc4-cdbe-49b0-9f42-c7cb70708721"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:34:18.803609 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.803501    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-kube-api-access-gs585" (OuterVolumeSpecName: "kube-api-access-gs585") pod "a29c8cc4-cdbe-49b0-9f42-c7cb70708721" (UID: "a29c8cc4-cdbe-49b0-9f42-c7cb70708721"). InnerVolumeSpecName "kube-api-access-gs585". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:34:18.902532 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.902511    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs585\" (UniqueName: \"kubernetes.io/projected/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-kube-api-access-gs585\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:18.902532 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.902531    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-dshm\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:18.902681 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:18.902541    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29c8cc4-cdbe-49b0-9f42-c7cb70708721-tls-certs\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:19.638379 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.638352    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"
Jul 18 02:34:19.638803 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.638380    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w" event={"ID":"a29c8cc4-cdbe-49b0-9f42-c7cb70708721","Type":"ContainerDied","Data":"8f0b3cfde8185a98735784697102e30f4d369f637e2cd1168fe5b46b1cdfee91"}
Jul 18 02:34:19.638803 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.638430    2576 scope.go:117] "RemoveContainer" containerID="f2f6f79bccdc54cb5b28ca7900df92e782d1f6dc3e507254a3fdc4b722e5416e"
Jul 18 02:34:19.652848 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.652818    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"]
Jul 18 02:34:19.654506 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.654481    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9e2f67-d5ttr"]
Jul 18 02:34:19.657504 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.657479    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"]
Jul 18 02:34:19.662962 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.662938    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9e2f67/test-model-9e2f67-kserve-6f46c958dc-8qs8w"]
Jul 18 02:34:19.827889 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:19.827855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:34:21.082110 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:21.082069    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90354506-3ccf-43ff-a26b-5d973e774e61" path="/var/lib/kubelet/pods/90354506-3ccf-43ff-a26b-5d973e774e61/volumes"
Jul 18 02:34:21.082664 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:21.082641    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a29c8cc4-cdbe-49b0-9f42-c7cb70708721" path="/var/lib/kubelet/pods/a29c8cc4-cdbe-49b0-9f42-c7cb70708721/volumes"
Jul 18 02:34:26.397740 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.397706    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"]
Jul 18 02:34:26.398115 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.398089    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29c8cc4-cdbe-49b0-9f42-c7cb70708721" containerName="main"
Jul 18 02:34:26.398115 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.398100    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29c8cc4-cdbe-49b0-9f42-c7cb70708721" containerName="main"
Jul 18 02:34:26.398115 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.398115    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90354506-3ccf-43ff-a26b-5d973e774e61" containerName="revoke-keys"
Jul 18 02:34:26.398254 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.398121    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90354506-3ccf-43ff-a26b-5d973e774e61" containerName="revoke-keys"
Jul 18 02:34:26.398254 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.398179    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90354506-3ccf-43ff-a26b-5d973e774e61" containerName="revoke-keys"
Jul 18 02:34:26.398254 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.398201    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a29c8cc4-cdbe-49b0-9f42-c7cb70708721" containerName="main"
Jul 18 02:34:26.401350 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.401332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:26.403945 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.403921    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:34:26.404083 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.403963    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:34:26.409347 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.409321    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"]
Jul 18 02:34:26.465111 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.465085    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86\" (UID: \"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:26.565849 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.565824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86\" (UID: \"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:26.566427 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.566410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86\" (UID: \"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:26.713857 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.713832    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:26.844734 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:26.844715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"]
Jul 18 02:34:26.847084 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:34:26.847054    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf4e5b6_4b5d_420e_a5d4_b01f8c711581.slice/crio-b003ffb5c70dc568846cfb495350421ca26d208b6213b1b282d4ffcce3a467d9 WatchSource:0}: Error finding container b003ffb5c70dc568846cfb495350421ca26d208b6213b1b282d4ffcce3a467d9: Status 404 returned error can't find the container with id b003ffb5c70dc568846cfb495350421ca26d208b6213b1b282d4ffcce3a467d9
Jul 18 02:34:27.681342 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:27.681308    2576 generic.go:356] "Generic (PLEG): container finished" podID="aaf4e5b6-4b5d-420e-a5d4-b01f8c711581" containerID="7313e2c1312b92f985ff428f08c5546304a17cefc01b80bcd6dd1bb908ece4b2" exitCode=0
Jul 18 02:34:27.681676 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:27.681343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86" event={"ID":"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581","Type":"ContainerDied","Data":"7313e2c1312b92f985ff428f08c5546304a17cefc01b80bcd6dd1bb908ece4b2"}
Jul 18 02:34:27.681676 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:27.681395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86" event={"ID":"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581","Type":"ContainerStarted","Data":"b003ffb5c70dc568846cfb495350421ca26d208b6213b1b282d4ffcce3a467d9"}
Jul 18 02:34:28.814186 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:28.814164    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:28.887926 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:28.887893    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581-maas-api-service-ca\") pod \"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581\" (UID: \"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581\") "
Jul 18 02:34:28.888226 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:28.888183    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aaf4e5b6-4b5d-420e-a5d4-b01f8c711581" (UID: "aaf4e5b6-4b5d-420e-a5d4-b01f8c711581"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:34:28.989485 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:28.989458    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581-maas-api-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:34:29.692050 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:29.692012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86" event={"ID":"aaf4e5b6-4b5d-420e-a5d4-b01f8c711581","Type":"ContainerDied","Data":"b003ffb5c70dc568846cfb495350421ca26d208b6213b1b282d4ffcce3a467d9"}
Jul 18 02:34:29.692050 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:29.692028    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"
Jul 18 02:34:29.692353 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:29.692062    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b003ffb5c70dc568846cfb495350421ca26d208b6213b1b282d4ffcce3a467d9"
Jul 18 02:34:39.726122 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:39.726082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:34:41.471095 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:41.471060    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"]
Jul 18 02:34:41.474399 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:41.474376    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a45ea2-4dr86"]
Jul 18 02:34:43.080748 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:43.080714    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaf4e5b6-4b5d-420e-a5d4-b01f8c711581" path="/var/lib/kubelet/pods/aaf4e5b6-4b5d-420e-a5d4-b01f8c711581/volumes"
Jul 18 02:34:59.594863 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.594827    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"]
Jul 18 02:34:59.595476 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.595284    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf4e5b6-4b5d-420e-a5d4-b01f8c711581" containerName="revoke-keys"
Jul 18 02:34:59.595476 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.595303    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf4e5b6-4b5d-420e-a5d4-b01f8c711581" containerName="revoke-keys"
Jul 18 02:34:59.595476 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.595416    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf4e5b6-4b5d-420e-a5d4-b01f8c711581" containerName="revoke-keys"
Jul 18 02:34:59.599022 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.598998    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.601936 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.601914    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:34:59.602057 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.601917    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rbxdz\""
Jul 18 02:34:59.602057 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.601917    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e8ceff-serving-cert\""
Jul 18 02:34:59.603024 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.603005    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:34:59.608183 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.608102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"]
Jul 18 02:34:59.651427 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.651394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:34:59.741524 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.741492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.741692 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.741560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03226e2-31c7-49a1-8ee2-3269a1dabb50-openshift-service-ca\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.741692 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.741614    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st5mh\" (UniqueName: \"kubernetes.io/projected/a03226e2-31c7-49a1-8ee2-3269a1dabb50-kube-api-access-st5mh\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.842170 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.842136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.842383 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.842236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03226e2-31c7-49a1-8ee2-3269a1dabb50-openshift-service-ca\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.842383 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:34:59.842292    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-serving-cert: secret "maas-api-e2e-api-b-e8ceff-serving-cert" not found
Jul 18 02:34:59.842383 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:34:59.842362    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls podName:a03226e2-31c7-49a1-8ee2-3269a1dabb50 nodeName:}" failed. No retries permitted until 2026-07-18 02:35:00.342346691 +0000 UTC m=+2274.834990499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls") pod "maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" (UID: "a03226e2-31c7-49a1-8ee2-3269a1dabb50") : secret "maas-api-e2e-api-b-e8ceff-serving-cert" not found
Jul 18 02:34:59.842383 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.842294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5mh\" (UniqueName: \"kubernetes.io/projected/a03226e2-31c7-49a1-8ee2-3269a1dabb50-kube-api-access-st5mh\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.842828 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.842811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03226e2-31c7-49a1-8ee2-3269a1dabb50-openshift-service-ca\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:34:59.852818 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:34:59.852759    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st5mh\" (UniqueName: \"kubernetes.io/projected/a03226e2-31c7-49a1-8ee2-3269a1dabb50-kube-api-access-st5mh\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:00.348214 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:00.348169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:00.350662 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:00.350642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls\") pod \"maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:00.512143 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:00.512112    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:00.647222 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:00.647178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"]
Jul 18 02:35:00.649780 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:35:00.649753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03226e2_31c7_49a1_8ee2_3269a1dabb50.slice/crio-c3fda92abfd122dc479ccf5bc50f32ad8d06385b5d0032b19fb9823a0e046984 WatchSource:0}: Error finding container c3fda92abfd122dc479ccf5bc50f32ad8d06385b5d0032b19fb9823a0e046984: Status 404 returned error can't find the container with id c3fda92abfd122dc479ccf5bc50f32ad8d06385b5d0032b19fb9823a0e046984
Jul 18 02:35:00.845620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:00.845594    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" event={"ID":"a03226e2-31c7-49a1-8ee2-3269a1dabb50","Type":"ContainerStarted","Data":"c3fda92abfd122dc479ccf5bc50f32ad8d06385b5d0032b19fb9823a0e046984"}
Jul 18 02:35:03.863372 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:03.863336    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" event={"ID":"a03226e2-31c7-49a1-8ee2-3269a1dabb50","Type":"ContainerStarted","Data":"511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26"}
Jul 18 02:35:03.863864 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:03.863477    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:03.881240 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:03.881156    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" podStartSLOduration=2.689612282 podStartE2EDuration="4.881143228s" totalImagesPullingTime="2.191530946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:35:03.879967868 +0000 UTC m=+2278.372611695" watchObservedRunningTime="2026-07-18 02:35:03.881143228 +0000 UTC m=+2278.373787053"
Jul 18 02:35:09.873064 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:09.873034    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:16.629134 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:16.629096    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:35:35.228683 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:35.228641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:35:59.082701 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.082625    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"]
Jul 18 02:35:59.083058 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.082889    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" podUID="a03226e2-31c7-49a1-8ee2-3269a1dabb50" containerName="maas-api" containerID="cri-o://511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26" gracePeriod=30
Jul 18 02:35:59.325523 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.325500    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:35:59.336387 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.336337    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st5mh\" (UniqueName: \"kubernetes.io/projected/a03226e2-31c7-49a1-8ee2-3269a1dabb50-kube-api-access-st5mh\") pod \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") "
Jul 18 02:35:59.336496 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.336395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03226e2-31c7-49a1-8ee2-3269a1dabb50-openshift-service-ca\") pod \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") "
Jul 18 02:35:59.336496 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.336479    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls\") pod \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\" (UID: \"a03226e2-31c7-49a1-8ee2-3269a1dabb50\") "
Jul 18 02:35:59.336732 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.336706    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a03226e2-31c7-49a1-8ee2-3269a1dabb50-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a03226e2-31c7-49a1-8ee2-3269a1dabb50" (UID: "a03226e2-31c7-49a1-8ee2-3269a1dabb50"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:35:59.338459 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.338436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a03226e2-31c7-49a1-8ee2-3269a1dabb50" (UID: "a03226e2-31c7-49a1-8ee2-3269a1dabb50"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:35:59.338541 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.338494    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a03226e2-31c7-49a1-8ee2-3269a1dabb50-kube-api-access-st5mh" (OuterVolumeSpecName: "kube-api-access-st5mh") pod "a03226e2-31c7-49a1-8ee2-3269a1dabb50" (UID: "a03226e2-31c7-49a1-8ee2-3269a1dabb50"). InnerVolumeSpecName "kube-api-access-st5mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:35:59.437596 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.437562    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st5mh\" (UniqueName: \"kubernetes.io/projected/a03226e2-31c7-49a1-8ee2-3269a1dabb50-kube-api-access-st5mh\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:35:59.437596 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.437593    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03226e2-31c7-49a1-8ee2-3269a1dabb50-openshift-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:35:59.437596 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:35:59.437604    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03226e2-31c7-49a1-8ee2-3269a1dabb50-maas-api-tls\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:36:00.131656 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.131622    2576 generic.go:356] "Generic (PLEG): container finished" podID="a03226e2-31c7-49a1-8ee2-3269a1dabb50" containerID="511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26" exitCode=0
Jul 18 02:36:00.132103 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.131661    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"
Jul 18 02:36:00.132103 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.131700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" event={"ID":"a03226e2-31c7-49a1-8ee2-3269a1dabb50","Type":"ContainerDied","Data":"511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26"}
Jul 18 02:36:00.132103 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.131751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf" event={"ID":"a03226e2-31c7-49a1-8ee2-3269a1dabb50","Type":"ContainerDied","Data":"c3fda92abfd122dc479ccf5bc50f32ad8d06385b5d0032b19fb9823a0e046984"}
Jul 18 02:36:00.132103 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.131775    2576 scope.go:117] "RemoveContainer" containerID="511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26"
Jul 18 02:36:00.140987 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.140964    2576 scope.go:117] "RemoveContainer" containerID="511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26"
Jul 18 02:36:00.141273 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:36:00.141256    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26\": container with ID starting with 511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26 not found: ID does not exist" containerID="511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26"
Jul 18 02:36:00.141373 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.141282    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26"} err="failed to get container status \"511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26\": rpc error: code = NotFound desc = could not find container \"511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26\": container with ID starting with 511224bd6192cf8d9fe99d4feb9ba54b001238b0447d6d7c7322d6a7c5428f26 not found: ID does not exist"
Jul 18 02:36:00.153087 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.153065    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"]
Jul 18 02:36:00.156970 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:00.156946    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8ceff-6bb6bfd5bb-hz5rf"]
Jul 18 02:36:01.080463 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:01.080428    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a03226e2-31c7-49a1-8ee2-3269a1dabb50" path="/var/lib/kubelet/pods/a03226e2-31c7-49a1-8ee2-3269a1dabb50/volumes"
Jul 18 02:36:04.322781 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:04.322747    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:36:14.122744 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.122705    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"]
Jul 18 02:36:14.123306 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.123286    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a03226e2-31c7-49a1-8ee2-3269a1dabb50" containerName="maas-api"
Jul 18 02:36:14.123457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.123308    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a03226e2-31c7-49a1-8ee2-3269a1dabb50" containerName="maas-api"
Jul 18 02:36:14.123457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.123419    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a03226e2-31c7-49a1-8ee2-3269a1dabb50" containerName="maas-api"
Jul 18 02:36:14.126403 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.126381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.129171 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.129150    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rbxdz\""
Jul 18 02:36:14.129293 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.129270    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:36:14.130224 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.130188    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0d6b43-serving-cert\""
Jul 18 02:36:14.130318 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.130226    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 18 02:36:14.138420 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.138399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"]
Jul 18 02:36:14.163181 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.163150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tntnc\" (UniqueName: \"kubernetes.io/projected/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-kube-api-access-tntnc\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.163349 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.163258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.163349 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.163336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.264543 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.264513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.264666 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.264560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tntnc\" (UniqueName: \"kubernetes.io/projected/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-kube-api-access-tntnc\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.264666 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.264619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.264750 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:36:14.264702    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-serving-cert: secret "maas-api-e2e-shared-a-0d6b43-serving-cert" not found
Jul 18 02:36:14.264795 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:36:14.264771    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls podName:7f1000d8-fc4d-4192-aedb-869d0f1f17a2 nodeName:}" failed. No retries permitted until 2026-07-18 02:36:14.764755502 +0000 UTC m=+2349.257399306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls") pod "maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" (UID: "7f1000d8-fc4d-4192-aedb-869d0f1f17a2") : secret "maas-api-e2e-shared-a-0d6b43-serving-cert" not found
Jul 18 02:36:14.265097 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.265080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.273546 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.273521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tntnc\" (UniqueName: \"kubernetes.io/projected/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-kube-api-access-tntnc\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.769096 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.769061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:14.771457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:14.771433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls\") pod \"maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:15.038137 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:15.038049    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:15.170213 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:15.170168    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"]
Jul 18 02:36:15.172097 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:36:15.172062    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f1000d8_fc4d_4192_aedb_869d0f1f17a2.slice/crio-ad34a4d64f9d08ca35433c55b427b88ba509dc921f55ac2f9eae08c507f9ae09 WatchSource:0}: Error finding container ad34a4d64f9d08ca35433c55b427b88ba509dc921f55ac2f9eae08c507f9ae09: Status 404 returned error can't find the container with id ad34a4d64f9d08ca35433c55b427b88ba509dc921f55ac2f9eae08c507f9ae09
Jul 18 02:36:15.206232 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:15.206180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" event={"ID":"7f1000d8-fc4d-4192-aedb-869d0f1f17a2","Type":"ContainerStarted","Data":"ad34a4d64f9d08ca35433c55b427b88ba509dc921f55ac2f9eae08c507f9ae09"}
Jul 18 02:36:16.213249 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:16.213219    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" event={"ID":"7f1000d8-fc4d-4192-aedb-869d0f1f17a2","Type":"ContainerStarted","Data":"8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138"}
Jul 18 02:36:16.213660 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:16.213342    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:16.231372 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:16.231328    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" podStartSLOduration=1.823511092 podStartE2EDuration="2.231316443s" totalImagesPullingTime="407.805351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:36:16.229732854 +0000 UTC m=+2350.722376680" watchObservedRunningTime="2026-07-18 02:36:16.231316443 +0000 UTC m=+2350.723960277"
Jul 18 02:36:19.923134 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:19.923103    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:36:22.222716 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:22.222691    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:36:34.223555 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:34.223529    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:36:42.423090 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.423060    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"]
Jul 18 02:36:42.426421 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.426398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.430078 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.430054    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0d6b43\"/\"kube-root-ca.crt\""
Jul 18 02:36:42.430246 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.430142    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d6b43\"/\"auth-test-model-0d6b43-kserve-self-signed-certs\""
Jul 18 02:36:42.430246 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.430145    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d6b43\"/\"default-dockercfg-9b5bx\""
Jul 18 02:36:42.430246 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.430231    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0d6b43\"/\"openshift-service-ca.crt\""
Jul 18 02:36:42.436748 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.436728    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"]
Jul 18 02:36:42.598088 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.598059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tls-certs\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.598266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.598092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-home\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.598266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.598118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-model-cache\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.598266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.598207    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-dshm\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.598391 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.598264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tblwr\" (UniqueName: \"kubernetes.io/projected/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-kube-api-access-tblwr\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.598391 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.598300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tmp-dir\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.699629 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.699602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-model-cache\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.699795 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.699648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-dshm\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.699795 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.699772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tblwr\" (UniqueName: \"kubernetes.io/projected/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-kube-api-access-tblwr\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.699912 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.699828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tmp-dir\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.699912 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.699878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tls-certs\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.699912 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.699908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-home\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.700097 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.700074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-model-cache\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.700170 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.700148    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tmp-dir\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.700305 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.700283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-home\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.701719 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.701701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-dshm\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.702128 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.702110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tls-certs\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.707410 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.707392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tblwr\" (UniqueName: \"kubernetes.io/projected/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-kube-api-access-tblwr\") pod \"auth-test-model-0d6b43-kserve-6855c87cf9-sss92\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") " pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.736363 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.736346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:42.866139 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:42.866114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"]
Jul 18 02:36:42.868743 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:36:42.868718    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c84bbff_7445_4b2f_bbb9_0ab47c6b9405.slice/crio-1a4f9ba38fa05d7d31bfbba3b0997ccbbc4d5250ac51346d73d82128685c09d2 WatchSource:0}: Error finding container 1a4f9ba38fa05d7d31bfbba3b0997ccbbc4d5250ac51346d73d82128685c09d2: Status 404 returned error can't find the container with id 1a4f9ba38fa05d7d31bfbba3b0997ccbbc4d5250ac51346d73d82128685c09d2
Jul 18 02:36:43.347823 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:43.347785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92" event={"ID":"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405","Type":"ContainerStarted","Data":"8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6"}
Jul 18 02:36:43.347984 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:43.347837    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92" event={"ID":"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405","Type":"ContainerStarted","Data":"1a4f9ba38fa05d7d31bfbba3b0997ccbbc4d5250ac51346d73d82128685c09d2"}
Jul 18 02:36:43.365766 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:43.365715    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92" podStartSLOduration=1.365702027 podStartE2EDuration="1.365702027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:36:43.363024997 +0000 UTC m=+2377.855668823" watchObservedRunningTime="2026-07-18 02:36:43.365702027 +0000 UTC m=+2377.858345910"
Jul 18 02:36:46.721814 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:46.721776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:36:52.736815 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:52.736777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:52.737213 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:52.736825    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:52.749079 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:52.749055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:36:53.403536 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:36:53.403509    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:37:12.423035 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:37:12.423006    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:37:30.620642 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:37:30.620556    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:37:48.032486 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:37:48.032447    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:38:03.129450 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:38:03.129417    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:38:23.021892 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:38:23.021854    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:38:42.123405 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:38:42.123373    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:39:01.524128 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:01.524029    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:39:16.397388 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:16.397352    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"]
Jul 18 02:39:20.432572 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:20.432540    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:39:31.417580 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.417534    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92" podUID="2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" containerName="main" containerID="cri-o://8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6" gracePeriod=45
Jul 18 02:39:31.665008 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.664984    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:39:31.727728 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.727702    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"]
Jul 18 02:39:31.728086 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.728075    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" containerName="main"
Jul 18 02:39:31.728130 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.728088    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" containerName="main"
Jul 18 02:39:31.728166 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.728152    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" containerName="main"
Jul 18 02:39:31.731111 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.731093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.733644 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.733624    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d6b43\"/\"sub-test-model-0d6b43-kserve-self-signed-certs\""
Jul 18 02:39:31.745481 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.745446    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"]
Jul 18 02:39:31.784012 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.783985    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-dshm\") pod \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") "
Jul 18 02:39:31.784120 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784095    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tls-certs\") pod \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") "
Jul 18 02:39:31.784182 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784137    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-home\") pod \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") "
Jul 18 02:39:31.784271 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tblwr\" (UniqueName: \"kubernetes.io/projected/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-kube-api-access-tblwr\") pod \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") "
Jul 18 02:39:31.784351 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784330    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tmp-dir\") pod \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") "
Jul 18 02:39:31.784532 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784512    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-model-cache\") pod \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\" (UID: \"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405\") "
Jul 18 02:39:31.784636 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784386    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-home" (OuterVolumeSpecName: "home") pod "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" (UID: "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:39:31.784636 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784568    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" (UID: "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:39:31.784842 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784815    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-home\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:39:31.784842 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784832    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tmp-dir\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:39:31.784842 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.784825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" (UID: "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:39:31.786317 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.786293    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-dshm" (OuterVolumeSpecName: "dshm") pod "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" (UID: "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:39:31.786396 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.786365    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" (UID: "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:39:31.786435 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.786391    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-kube-api-access-tblwr" (OuterVolumeSpecName: "kube-api-access-tblwr") pod "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" (UID: "2c84bbff-7445-4b2f-bbb9-0ab47c6b9405"). InnerVolumeSpecName "kube-api-access-tblwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:39:31.886285 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07e381fc-56ea-4225-9932-32a6b8996a52-tls-certs\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.886457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-model-cache\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.886457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886370    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-dshm\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.886457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-home\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.886457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886439    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8bk2\" (UniqueName: \"kubernetes.io/projected/07e381fc-56ea-4225-9932-32a6b8996a52-kube-api-access-w8bk2\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.886620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-tmp-dir\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.886620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886566    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-model-cache\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:39:31.886620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886576    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-dshm\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:39:31.886620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886585    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-tls-certs\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:39:31.886620 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.886593    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tblwr\" (UniqueName: \"kubernetes.io/projected/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405-kube-api-access-tblwr\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:39:31.987300 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-tmp-dir\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987300 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07e381fc-56ea-4225-9932-32a6b8996a52-tls-certs\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987505 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-model-cache\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987505 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-dshm\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987505 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-home\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987505 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8bk2\" (UniqueName: \"kubernetes.io/projected/07e381fc-56ea-4225-9932-32a6b8996a52-kube-api-access-w8bk2\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987713 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-tmp-dir\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987713 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-model-cache\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.987794 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.987763    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-home\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.989595 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.989575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-dshm\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.989803 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.989783    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07e381fc-56ea-4225-9932-32a6b8996a52-tls-certs\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:31.995652 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:31.995633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8bk2\" (UniqueName: \"kubernetes.io/projected/07e381fc-56ea-4225-9932-32a6b8996a52-kube-api-access-w8bk2\") pod \"sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") " pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:32.043664 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.043642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:32.171364 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.171317    2576 generic.go:356] "Generic (PLEG): container finished" podID="2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" containerID="8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6" exitCode=0
Jul 18 02:39:32.171528 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.171389    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"
Jul 18 02:39:32.171528 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.171391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92" event={"ID":"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405","Type":"ContainerDied","Data":"8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6"}
Jul 18 02:39:32.171528 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.171446    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92" event={"ID":"2c84bbff-7445-4b2f-bbb9-0ab47c6b9405","Type":"ContainerDied","Data":"1a4f9ba38fa05d7d31bfbba3b0997ccbbc4d5250ac51346d73d82128685c09d2"}
Jul 18 02:39:32.171528 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.171471    2576 scope.go:117] "RemoveContainer" containerID="8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6"
Jul 18 02:39:32.175010 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.174827    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"]
Jul 18 02:39:32.177867 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:39:32.177844    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07e381fc_56ea_4225_9932_32a6b8996a52.slice/crio-9130c5e64a1c010fc9f0584c20914a2ea92f14c285957e3e1c5c915cb6ebc81a WatchSource:0}: Error finding container 9130c5e64a1c010fc9f0584c20914a2ea92f14c285957e3e1c5c915cb6ebc81a: Status 404 returned error can't find the container with id 9130c5e64a1c010fc9f0584c20914a2ea92f14c285957e3e1c5c915cb6ebc81a
Jul 18 02:39:32.180691 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.180650    2576 scope.go:117] "RemoveContainer" containerID="8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6"
Jul 18 02:39:32.180995 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:39:32.180971    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6\": container with ID starting with 8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6 not found: ID does not exist" containerID="8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6"
Jul 18 02:39:32.181087 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.181002    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6"} err="failed to get container status \"8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6\": rpc error: code = NotFound desc = could not find container \"8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6\": container with ID starting with 8e4d6a480b448aa4ba1046b96fafa2016b14d0a9f8c51dd46715cab53500eaf6 not found: ID does not exist"
Jul 18 02:39:32.195367 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.195349    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"]
Jul 18 02:39:32.201500 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:32.201480    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/auth-test-model-0d6b43-kserve-6855c87cf9-sss92"]
Jul 18 02:39:33.080167 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:33.080138    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c84bbff-7445-4b2f-bbb9-0ab47c6b9405" path="/var/lib/kubelet/pods/2c84bbff-7445-4b2f-bbb9-0ab47c6b9405/volumes"
Jul 18 02:39:33.177884 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:33.177845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x" event={"ID":"07e381fc-56ea-4225-9932-32a6b8996a52","Type":"ContainerStarted","Data":"3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1"}
Jul 18 02:39:33.178062 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:33.177896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x" event={"ID":"07e381fc-56ea-4225-9932-32a6b8996a52","Type":"ContainerStarted","Data":"9130c5e64a1c010fc9f0584c20914a2ea92f14c285957e3e1c5c915cb6ebc81a"}
Jul 18 02:39:33.197708 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:33.197659    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x" podStartSLOduration=2.197643887 podStartE2EDuration="2.197643887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:39:33.19589571 +0000 UTC m=+2547.688539538" watchObservedRunningTime="2026-07-18 02:39:33.197643887 +0000 UTC m=+2547.690287752"
Jul 18 02:39:34.426094 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:34.426064    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:39:42.044783 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:42.044745    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:42.045228 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:42.044992    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:42.057398 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:42.057374    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:39:42.234768 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:39:42.234739    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:40:20.234244 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:40:20.234210    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:40:59.129679 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:40:59.129644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:41:06.691490 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:06.691452    2576 scope.go:117] "RemoveContainer" containerID="7313e2c1312b92f985ff428f08c5546304a17cefc01b80bcd6dd1bb908ece4b2"
Jul 18 02:41:06.700404 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:06.700382    2576 scope.go:117] "RemoveContainer" containerID="af36ed4f14c329687bb0a1e674f7739039dcae2799bb9de3feaff10b8c406ed5"
Jul 18 02:41:15.734203 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:15.734152    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:41:22.457043 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:22.457006    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"]
Jul 18 02:41:31.027391 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.027360    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"]
Jul 18 02:41:31.030975 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.030954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.033586 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.033566    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d6b43\"/\"rate-test-model-0d6b43-kserve-self-signed-certs\""
Jul 18 02:41:31.041490 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.041467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"]
Jul 18 02:41:31.151598 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.151565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-dshm\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.151733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.151601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjtln\" (UniqueName: \"kubernetes.io/projected/e63e721b-076f-4e5d-8744-671ceebafbc0-kube-api-access-rjtln\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.151733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.151624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-tmp-dir\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.151733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.151685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63e721b-076f-4e5d-8744-671ceebafbc0-tls-certs\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.151733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.151732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-model-cache\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.151875 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.151766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-home\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.252864 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.252835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-dshm\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.252979 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.252870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjtln\" (UniqueName: \"kubernetes.io/projected/e63e721b-076f-4e5d-8744-671ceebafbc0-kube-api-access-rjtln\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.252979 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.252898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-tmp-dir\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.252979 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.252933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63e721b-076f-4e5d-8744-671ceebafbc0-tls-certs\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.253077 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.252993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-model-cache\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.253077 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.253039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-home\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.253415 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.253393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-tmp-dir\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.253548 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.253417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-model-cache\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.253548 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.253444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-home\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.255332 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.255310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-dshm\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.255578 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.255560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63e721b-076f-4e5d-8744-671ceebafbc0-tls-certs\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.261175 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.261151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjtln\" (UniqueName: \"kubernetes.io/projected/e63e721b-076f-4e5d-8744-671ceebafbc0-kube-api-access-rjtln\") pod \"rate-test-model-0d6b43-kserve-b564d7d47-njjw6\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") " pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.330881 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.330823    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:41:31.342711 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.342686    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:31.473697 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.473665    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"]
Jul 18 02:41:31.476769 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:41:31.476736    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode63e721b_076f_4e5d_8744_671ceebafbc0.slice/crio-f7fade89e294ea62a147d234e4a27c8a0ed9108299a9dfa392117674b0dc9915 WatchSource:0}: Error finding container f7fade89e294ea62a147d234e4a27c8a0ed9108299a9dfa392117674b0dc9915: Status 404 returned error can't find the container with id f7fade89e294ea62a147d234e4a27c8a0ed9108299a9dfa392117674b0dc9915
Jul 18 02:41:31.755564 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.755531    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6" event={"ID":"e63e721b-076f-4e5d-8744-671ceebafbc0","Type":"ContainerStarted","Data":"2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a"}
Jul 18 02:41:31.755564 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.755574    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6" event={"ID":"e63e721b-076f-4e5d-8744-671ceebafbc0","Type":"ContainerStarted","Data":"f7fade89e294ea62a147d234e4a27c8a0ed9108299a9dfa392117674b0dc9915"}
Jul 18 02:41:31.771962 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:31.771911    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6" podStartSLOduration=0.77189171 podStartE2EDuration="771.89171ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:41:31.771522331 +0000 UTC m=+2666.264166157" watchObservedRunningTime="2026-07-18 02:41:31.77189171 +0000 UTC m=+2666.264535538"
Jul 18 02:41:37.478140 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.478102    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x" podUID="07e381fc-56ea-4225-9932-32a6b8996a52" containerName="main" containerID="cri-o://3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1" gracePeriod=45
Jul 18 02:41:37.733004 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.732947    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:41:37.784295 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.784264    2576 generic.go:356] "Generic (PLEG): container finished" podID="07e381fc-56ea-4225-9932-32a6b8996a52" containerID="3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1" exitCode=0
Jul 18 02:41:37.784429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.784303    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"
Jul 18 02:41:37.784429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.784331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x" event={"ID":"07e381fc-56ea-4225-9932-32a6b8996a52","Type":"ContainerDied","Data":"3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1"}
Jul 18 02:41:37.784429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.784372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x" event={"ID":"07e381fc-56ea-4225-9932-32a6b8996a52","Type":"ContainerDied","Data":"9130c5e64a1c010fc9f0584c20914a2ea92f14c285957e3e1c5c915cb6ebc81a"}
Jul 18 02:41:37.784429 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.784390    2576 scope.go:117] "RemoveContainer" containerID="3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1"
Jul 18 02:41:37.792603 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.792588    2576 scope.go:117] "RemoveContainer" containerID="3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1"
Jul 18 02:41:37.794714 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:41:37.794685    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1\": container with ID starting with 3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1 not found: ID does not exist" containerID="3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1"
Jul 18 02:41:37.794803 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.794714    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1"} err="failed to get container status \"3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1\": rpc error: code = NotFound desc = could not find container \"3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1\": container with ID starting with 3750c0374d15c95d8f7f494d2e2136c7c840e79b777348c54ea601694fe527a1 not found: ID does not exist"
Jul 18 02:41:37.909012 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.908984    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-dshm\") pod \"07e381fc-56ea-4225-9932-32a6b8996a52\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") "
Jul 18 02:41:37.909163 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909019    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-model-cache\") pod \"07e381fc-56ea-4225-9932-32a6b8996a52\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") "
Jul 18 02:41:37.909163 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909070    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8bk2\" (UniqueName: \"kubernetes.io/projected/07e381fc-56ea-4225-9932-32a6b8996a52-kube-api-access-w8bk2\") pod \"07e381fc-56ea-4225-9932-32a6b8996a52\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") "
Jul 18 02:41:37.909163 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909089    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07e381fc-56ea-4225-9932-32a6b8996a52-tls-certs\") pod \"07e381fc-56ea-4225-9932-32a6b8996a52\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") "
Jul 18 02:41:37.909163 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909160    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-home\") pod \"07e381fc-56ea-4225-9932-32a6b8996a52\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") "
Jul 18 02:41:37.909410 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909217    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-tmp-dir\") pod \"07e381fc-56ea-4225-9932-32a6b8996a52\" (UID: \"07e381fc-56ea-4225-9932-32a6b8996a52\") "
Jul 18 02:41:37.909410 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909341    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-model-cache" (OuterVolumeSpecName: "model-cache") pod "07e381fc-56ea-4225-9932-32a6b8996a52" (UID: "07e381fc-56ea-4225-9932-32a6b8996a52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:41:37.909521 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909494    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-home" (OuterVolumeSpecName: "home") pod "07e381fc-56ea-4225-9932-32a6b8996a52" (UID: "07e381fc-56ea-4225-9932-32a6b8996a52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:41:37.909571 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909511    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07e381fc-56ea-4225-9932-32a6b8996a52" (UID: "07e381fc-56ea-4225-9932-32a6b8996a52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:41:37.909618 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909577    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-model-cache\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:41:37.909618 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.909596    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-home\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:41:37.911242 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.911215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07e381fc-56ea-4225-9932-32a6b8996a52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07e381fc-56ea-4225-9932-32a6b8996a52" (UID: "07e381fc-56ea-4225-9932-32a6b8996a52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:41:37.911242 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.911232    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e381fc-56ea-4225-9932-32a6b8996a52-kube-api-access-w8bk2" (OuterVolumeSpecName: "kube-api-access-w8bk2") pod "07e381fc-56ea-4225-9932-32a6b8996a52" (UID: "07e381fc-56ea-4225-9932-32a6b8996a52"). InnerVolumeSpecName "kube-api-access-w8bk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:41:37.911436 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:37.911305    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-dshm" (OuterVolumeSpecName: "dshm") pod "07e381fc-56ea-4225-9932-32a6b8996a52" (UID: "07e381fc-56ea-4225-9932-32a6b8996a52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:41:38.010440 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:38.010377    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-tmp-dir\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:41:38.010440 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:38.010402    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07e381fc-56ea-4225-9932-32a6b8996a52-dshm\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:41:38.010440 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:38.010411    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8bk2\" (UniqueName: \"kubernetes.io/projected/07e381fc-56ea-4225-9932-32a6b8996a52-kube-api-access-w8bk2\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:41:38.010440 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:38.010420    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07e381fc-56ea-4225-9932-32a6b8996a52-tls-certs\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:41:38.105656 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:38.105627    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"]
Jul 18 02:41:38.109234 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:38.109210    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/sub-test-model-0d6b43-kserve-55f4669dd9-xkg7x"]
Jul 18 02:41:39.080135 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:39.080097    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07e381fc-56ea-4225-9932-32a6b8996a52" path="/var/lib/kubelet/pods/07e381fc-56ea-4225-9932-32a6b8996a52/volumes"
Jul 18 02:41:41.343262 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:41.343229    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:41.343698 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:41.343476    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:41.355508 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:41.355487    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:41.635985 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:41.635917    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:41:41.814858 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:41.814833    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:41:56.828584 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:41:56.828554    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:42:28.627040 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:42:28.626990    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:42:48.735520 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:42:48.735480    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:42:56.951752 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:42:56.951722    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"]
Jul 18 02:43:01.436290 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.436255    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-140492067861168"]
Jul 18 02:43:01.436736 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.436654    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e381fc-56ea-4225-9932-32a6b8996a52" containerName="main"
Jul 18 02:43:01.436736 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.436667    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e381fc-56ea-4225-9932-32a6b8996a52" containerName="main"
Jul 18 02:43:01.436736 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.436735    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07e381fc-56ea-4225-9932-32a6b8996a52" containerName="main"
Jul 18 02:43:01.439979 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.439962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:01.442522 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.442496    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpsmz\""
Jul 18 02:43:01.443521 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.443496    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492067861168"]
Jul 18 02:43:01.512260 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.512230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq7sr\" (UniqueName: \"kubernetes.io/projected/cd629bb0-b477-4e45-bc3f-db296004b18d-kube-api-access-vq7sr\") pod \"test-curl-7689-140492067861168\" (UID: \"cd629bb0-b477-4e45-bc3f-db296004b18d\") " pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:01.613372 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.613345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq7sr\" (UniqueName: \"kubernetes.io/projected/cd629bb0-b477-4e45-bc3f-db296004b18d-kube-api-access-vq7sr\") pod \"test-curl-7689-140492067861168\" (UID: \"cd629bb0-b477-4e45-bc3f-db296004b18d\") " pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:01.622062 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.622033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq7sr\" (UniqueName: \"kubernetes.io/projected/cd629bb0-b477-4e45-bc3f-db296004b18d-kube-api-access-vq7sr\") pod \"test-curl-7689-140492067861168\" (UID: \"cd629bb0-b477-4e45-bc3f-db296004b18d\") " pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:01.750839 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.750814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:01.876669 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.876643    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-140492067861168"]
Jul 18 02:43:01.879255 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:43:01.879230    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd629bb0_b477_4e45_bc3f_db296004b18d.slice/crio-77851841777b5bb92276f61406c435ccde2629e4905608594738f641bd2cde49 WatchSource:0}: Error finding container 77851841777b5bb92276f61406c435ccde2629e4905608594738f641bd2cde49: Status 404 returned error can't find the container with id 77851841777b5bb92276f61406c435ccde2629e4905608594738f641bd2cde49
Jul 18 02:43:01.880612 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:01.880592    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:43:02.176975 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:02.176895    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492067861168" event={"ID":"cd629bb0-b477-4e45-bc3f-db296004b18d","Type":"ContainerStarted","Data":"77851841777b5bb92276f61406c435ccde2629e4905608594738f641bd2cde49"}
Jul 18 02:43:03.183279 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:03.183155    2576 generic.go:356] "Generic (PLEG): container finished" podID="cd629bb0-b477-4e45-bc3f-db296004b18d" containerID="538625c0c1de31483c835ce57698852fd4623202f52a6a27452e6de31b955e22" exitCode=0
Jul 18 02:43:03.183279 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:03.183224    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-140492067861168" event={"ID":"cd629bb0-b477-4e45-bc3f-db296004b18d","Type":"ContainerDied","Data":"538625c0c1de31483c835ce57698852fd4623202f52a6a27452e6de31b955e22"}
Jul 18 02:43:04.319029 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.319009    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:04.353338 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.353316    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7689-140492067861168_cd629bb0-b477-4e45-bc3f-db296004b18d/test-curl-7689-140492067861168/0.log"
Jul 18 02:43:04.376889 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.376859    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-140492067861168"]
Jul 18 02:43:04.380518 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.380494    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-140492067861168"]
Jul 18 02:43:04.426082 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.426063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:43:04.441690 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.441637    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq7sr\" (UniqueName: \"kubernetes.io/projected/cd629bb0-b477-4e45-bc3f-db296004b18d-kube-api-access-vq7sr\") pod \"cd629bb0-b477-4e45-bc3f-db296004b18d\" (UID: \"cd629bb0-b477-4e45-bc3f-db296004b18d\") "
Jul 18 02:43:04.443614 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.443595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd629bb0-b477-4e45-bc3f-db296004b18d-kube-api-access-vq7sr" (OuterVolumeSpecName: "kube-api-access-vq7sr") pod "cd629bb0-b477-4e45-bc3f-db296004b18d" (UID: "cd629bb0-b477-4e45-bc3f-db296004b18d"). InnerVolumeSpecName "kube-api-access-vq7sr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:43:04.543245 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:04.543224    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq7sr\" (UniqueName: \"kubernetes.io/projected/cd629bb0-b477-4e45-bc3f-db296004b18d-kube-api-access-vq7sr\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:05.080180 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:05.080145    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd629bb0-b477-4e45-bc3f-db296004b18d" path="/var/lib/kubelet/pods/cd629bb0-b477-4e45-bc3f-db296004b18d/volumes"
Jul 18 02:43:05.193681 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:05.193658    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-140492067861168"
Jul 18 02:43:05.193681 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:05.193676    2576 scope.go:117] "RemoveContainer" containerID="538625c0c1de31483c835ce57698852fd4623202f52a6a27452e6de31b955e22"
Jul 18 02:43:11.971927 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:11.971887    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6" podUID="e63e721b-076f-4e5d-8744-671ceebafbc0" containerName="main" containerID="cri-o://2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a" gracePeriod=45
Jul 18 02:43:12.219731 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.219710    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:43:12.228432 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.228371    2576 generic.go:356] "Generic (PLEG): container finished" podID="e63e721b-076f-4e5d-8744-671ceebafbc0" containerID="2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a" exitCode=0
Jul 18 02:43:12.228432 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.228419    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"
Jul 18 02:43:12.228546 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.228454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6" event={"ID":"e63e721b-076f-4e5d-8744-671ceebafbc0","Type":"ContainerDied","Data":"2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a"}
Jul 18 02:43:12.228546 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.228494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6" event={"ID":"e63e721b-076f-4e5d-8744-671ceebafbc0","Type":"ContainerDied","Data":"f7fade89e294ea62a147d234e4a27c8a0ed9108299a9dfa392117674b0dc9915"}
Jul 18 02:43:12.228546 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.228513    2576 scope.go:117] "RemoveContainer" containerID="2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a"
Jul 18 02:43:12.237733 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.237710    2576 scope.go:117] "RemoveContainer" containerID="2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a"
Jul 18 02:43:12.238019 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:43:12.237999    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a\": container with ID starting with 2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a not found: ID does not exist" containerID="2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a"
Jul 18 02:43:12.238106 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.238026    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a"} err="failed to get container status \"2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a\": rpc error: code = NotFound desc = could not find container \"2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a\": container with ID starting with 2246213d2c14cfa66712162428d6e7e4326ee1f486e5bc7efd2f93f23f6f887a not found: ID does not exist"
Jul 18 02:43:12.308678 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308658    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-dshm\") pod \"e63e721b-076f-4e5d-8744-671ceebafbc0\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") "
Jul 18 02:43:12.308776 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-tmp-dir\") pod \"e63e721b-076f-4e5d-8744-671ceebafbc0\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") "
Jul 18 02:43:12.308776 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63e721b-076f-4e5d-8744-671ceebafbc0-tls-certs\") pod \"e63e721b-076f-4e5d-8744-671ceebafbc0\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") "
Jul 18 02:43:12.308776 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308773    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-model-cache\") pod \"e63e721b-076f-4e5d-8744-671ceebafbc0\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") "
Jul 18 02:43:12.308899 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-home\") pod \"e63e721b-076f-4e5d-8744-671ceebafbc0\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") "
Jul 18 02:43:12.308899 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308886    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjtln\" (UniqueName: \"kubernetes.io/projected/e63e721b-076f-4e5d-8744-671ceebafbc0-kube-api-access-rjtln\") pod \"e63e721b-076f-4e5d-8744-671ceebafbc0\" (UID: \"e63e721b-076f-4e5d-8744-671ceebafbc0\") "
Jul 18 02:43:12.309010 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.308947    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e63e721b-076f-4e5d-8744-671ceebafbc0" (UID: "e63e721b-076f-4e5d-8744-671ceebafbc0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:43:12.309124 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.309087    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-model-cache" (OuterVolumeSpecName: "model-cache") pod "e63e721b-076f-4e5d-8744-671ceebafbc0" (UID: "e63e721b-076f-4e5d-8744-671ceebafbc0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:43:12.309124 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.309115    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-home" (OuterVolumeSpecName: "home") pod "e63e721b-076f-4e5d-8744-671ceebafbc0" (UID: "e63e721b-076f-4e5d-8744-671ceebafbc0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:43:12.309270 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.309251    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-tmp-dir\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:12.309318 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.309271    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-model-cache\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:12.309318 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.309284    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-home\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:12.310716 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.310690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-dshm" (OuterVolumeSpecName: "dshm") pod "e63e721b-076f-4e5d-8744-671ceebafbc0" (UID: "e63e721b-076f-4e5d-8744-671ceebafbc0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:43:12.310800 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.310773    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e63e721b-076f-4e5d-8744-671ceebafbc0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e63e721b-076f-4e5d-8744-671ceebafbc0" (UID: "e63e721b-076f-4e5d-8744-671ceebafbc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:43:12.310896 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.310877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e63e721b-076f-4e5d-8744-671ceebafbc0-kube-api-access-rjtln" (OuterVolumeSpecName: "kube-api-access-rjtln") pod "e63e721b-076f-4e5d-8744-671ceebafbc0" (UID: "e63e721b-076f-4e5d-8744-671ceebafbc0"). InnerVolumeSpecName "kube-api-access-rjtln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:43:12.410551 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.410515    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjtln\" (UniqueName: \"kubernetes.io/projected/e63e721b-076f-4e5d-8744-671ceebafbc0-kube-api-access-rjtln\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:12.410551 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.410546    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63e721b-076f-4e5d-8744-671ceebafbc0-dshm\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:12.410551 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.410556    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63e721b-076f-4e5d-8744-671ceebafbc0-tls-certs\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:12.552288 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.552261    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"]
Jul 18 02:43:12.555688 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:12.555667    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0d6b43/rate-test-model-0d6b43-kserve-b564d7d47-njjw6"]
Jul 18 02:43:13.081003 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:13.080972    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e63e721b-076f-4e5d-8744-671ceebafbc0" path="/var/lib/kubelet/pods/e63e721b-076f-4e5d-8744-671ceebafbc0/volumes"
Jul 18 02:43:26.967250 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:26.967210    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"]
Jul 18 02:43:26.967710 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:26.967473    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" podUID="7f1000d8-fc4d-4192-aedb-869d0f1f17a2" containerName="maas-api" containerID="cri-o://8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138" gracePeriod=30
Jul 18 02:43:27.213475 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.213449    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:43:27.306800 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.306729    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f1000d8-fc4d-4192-aedb-869d0f1f17a2" containerID="8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138" exitCode=0
Jul 18 02:43:27.306800 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.306780    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"
Jul 18 02:43:27.306800 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.306773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" event={"ID":"7f1000d8-fc4d-4192-aedb-869d0f1f17a2","Type":"ContainerDied","Data":"8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138"}
Jul 18 02:43:27.307005 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.306835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns" event={"ID":"7f1000d8-fc4d-4192-aedb-869d0f1f17a2","Type":"ContainerDied","Data":"ad34a4d64f9d08ca35433c55b427b88ba509dc921f55ac2f9eae08c507f9ae09"}
Jul 18 02:43:27.307005 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.306860    2576 scope.go:117] "RemoveContainer" containerID="8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138"
Jul 18 02:43:27.315260 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.315235    2576 scope.go:117] "RemoveContainer" containerID="8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138"
Jul 18 02:43:27.315517 ip-10-0-133-113 kubenswrapper[2576]: E0718 02:43:27.315497    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138\": container with ID starting with 8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138 not found: ID does not exist" containerID="8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138"
Jul 18 02:43:27.315584 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.315528    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138"} err="failed to get container status \"8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138\": rpc error: code = NotFound desc = could not find container \"8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138\": container with ID starting with 8e5a48c00fc5e484302fe47464e6828e895efdc543e399ae3d78b51924691138 not found: ID does not exist"
Jul 18 02:43:27.327926 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.327908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls\") pod \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") "
Jul 18 02:43:27.328012 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.328000    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tntnc\" (UniqueName: \"kubernetes.io/projected/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-kube-api-access-tntnc\") pod \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") "
Jul 18 02:43:27.328054 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.328042    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-openshift-service-ca\") pod \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\" (UID: \"7f1000d8-fc4d-4192-aedb-869d0f1f17a2\") "
Jul 18 02:43:27.328450 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.328428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f1000d8-fc4d-4192-aedb-869d0f1f17a2" (UID: "7f1000d8-fc4d-4192-aedb-869d0f1f17a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:43:27.329799 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.329774    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f1000d8-fc4d-4192-aedb-869d0f1f17a2" (UID: "7f1000d8-fc4d-4192-aedb-869d0f1f17a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:43:27.329910 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.329894    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-kube-api-access-tntnc" (OuterVolumeSpecName: "kube-api-access-tntnc") pod "7f1000d8-fc4d-4192-aedb-869d0f1f17a2" (UID: "7f1000d8-fc4d-4192-aedb-869d0f1f17a2"). InnerVolumeSpecName "kube-api-access-tntnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:43:27.429499 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.429475    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tntnc\" (UniqueName: \"kubernetes.io/projected/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-kube-api-access-tntnc\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:27.429499 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.429496    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-openshift-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:27.429622 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.429506    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f1000d8-fc4d-4192-aedb-869d0f1f17a2-maas-api-tls\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:27.628457 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.628431    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"]
Jul 18 02:43:27.633048 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:27.633029    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0d6b43-6f486f7988-qf8ns"]
Jul 18 02:43:29.081145 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:29.081068    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f1000d8-fc4d-4192-aedb-869d0f1f17a2" path="/var/lib/kubelet/pods/7f1000d8-fc4d-4192-aedb-869d0f1f17a2/volumes"
Jul 18 02:43:30.127522 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:30.127488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:43:38.324212 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324152    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"]
Jul 18 02:43:38.324782 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324704    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e63e721b-076f-4e5d-8744-671ceebafbc0" containerName="main"
Jul 18 02:43:38.324782 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324723    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e63e721b-076f-4e5d-8744-671ceebafbc0" containerName="main"
Jul 18 02:43:38.324782 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324759    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd629bb0-b477-4e45-bc3f-db296004b18d" containerName="test-curl-7689-140492067861168"
Jul 18 02:43:38.324782 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324769    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd629bb0-b477-4e45-bc3f-db296004b18d" containerName="test-curl-7689-140492067861168"
Jul 18 02:43:38.324782 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324781    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1000d8-fc4d-4192-aedb-869d0f1f17a2" containerName="maas-api"
Jul 18 02:43:38.325045 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324790    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1000d8-fc4d-4192-aedb-869d0f1f17a2" containerName="maas-api"
Jul 18 02:43:38.325045 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324875    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e63e721b-076f-4e5d-8744-671ceebafbc0" containerName="main"
Jul 18 02:43:38.325045 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324892    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f1000d8-fc4d-4192-aedb-869d0f1f17a2" containerName="maas-api"
Jul 18 02:43:38.325045 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.324901    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd629bb0-b477-4e45-bc3f-db296004b18d" containerName="test-curl-7689-140492067861168"
Jul 18 02:43:38.329777 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.329755    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:38.332869 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.332685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69gp\""
Jul 18 02:43:38.333183 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.333161    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:43:38.333790 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.333770    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"]
Jul 18 02:43:38.415537 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.415503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db9b52e2-329c-4441-89cf-3c59adc4ee8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw\" (UID: \"db9b52e2-329c-4441-89cf-3c59adc4ee8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:38.515976 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.515946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db9b52e2-329c-4441-89cf-3c59adc4ee8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw\" (UID: \"db9b52e2-329c-4441-89cf-3c59adc4ee8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:38.516593 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.516574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db9b52e2-329c-4441-89cf-3c59adc4ee8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw\" (UID: \"db9b52e2-329c-4441-89cf-3c59adc4ee8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:38.641798 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.641711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:38.773253 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:38.773228    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"]
Jul 18 02:43:38.775533 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:43:38.775498    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9b52e2_329c_4441_89cf_3c59adc4ee8c.slice/crio-75188cbe65a558579fee5e4ca5a7c3c94109b8f75681ac4e4410f75967d3a50d WatchSource:0}: Error finding container 75188cbe65a558579fee5e4ca5a7c3c94109b8f75681ac4e4410f75967d3a50d: Status 404 returned error can't find the container with id 75188cbe65a558579fee5e4ca5a7c3c94109b8f75681ac4e4410f75967d3a50d
Jul 18 02:43:39.368033 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:39.368005    2576 generic.go:356] "Generic (PLEG): container finished" podID="db9b52e2-329c-4441-89cf-3c59adc4ee8c" containerID="da9f6e34865264f9894aa95b5060c7c77846febea73fb37aabdc3399d68a473d" exitCode=0
Jul 18 02:43:39.368422 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:39.368080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw" event={"ID":"db9b52e2-329c-4441-89cf-3c59adc4ee8c","Type":"ContainerDied","Data":"da9f6e34865264f9894aa95b5060c7c77846febea73fb37aabdc3399d68a473d"}
Jul 18 02:43:39.368422 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:39.368128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw" event={"ID":"db9b52e2-329c-4441-89cf-3c59adc4ee8c","Type":"ContainerStarted","Data":"75188cbe65a558579fee5e4ca5a7c3c94109b8f75681ac4e4410f75967d3a50d"}
Jul 18 02:43:40.499161 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:40.499137    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:40.534915 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:40.534886    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db9b52e2-329c-4441-89cf-3c59adc4ee8c-maas-api-service-ca\") pod \"db9b52e2-329c-4441-89cf-3c59adc4ee8c\" (UID: \"db9b52e2-329c-4441-89cf-3c59adc4ee8c\") "
Jul 18 02:43:40.535200 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:40.535168    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db9b52e2-329c-4441-89cf-3c59adc4ee8c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "db9b52e2-329c-4441-89cf-3c59adc4ee8c" (UID: "db9b52e2-329c-4441-89cf-3c59adc4ee8c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:43:40.535295 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:40.535275    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db9b52e2-329c-4441-89cf-3c59adc4ee8c-maas-api-service-ca\") on node \"ip-10-0-133-113.ec2.internal\" DevicePath \"\""
Jul 18 02:43:41.379093 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:41.379023    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"
Jul 18 02:43:41.379266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:41.379012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw" event={"ID":"db9b52e2-329c-4441-89cf-3c59adc4ee8c","Type":"ContainerDied","Data":"75188cbe65a558579fee5e4ca5a7c3c94109b8f75681ac4e4410f75967d3a50d"}
Jul 18 02:43:41.379266 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:41.379142    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75188cbe65a558579fee5e4ca5a7c3c94109b8f75681ac4e4410f75967d3a50d"
Jul 18 02:43:50.630249 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:50.630211    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s4hzq"]
Jul 18 02:43:53.447891 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:53.447857    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"]
Jul 18 02:43:53.450625 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:53.450603    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-922cd9-5vzxw"]
Jul 18 02:43:55.081057 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:43:55.081005    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db9b52e2-329c-4441-89cf-3c59adc4ee8c" path="/var/lib/kubelet/pods/db9b52e2-329c-4441-89cf-3c59adc4ee8c/volumes"
Jul 18 02:44:07.124446 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:07.124373    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-rczg9_7cb54667-9934-49a0-8753-8c2d8f5abc15/copy-manifests/0.log"
Jul 18 02:44:07.143974 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:07.143940    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-rczg9_7cb54667-9934-49a0-8753-8c2d8f5abc15/manager/0.log"
Jul 18 02:44:09.397100 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:09.397070    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-w8h5t_9f61267b-b93b-45b4-8332-e0a31416924c/kuadrant-console-plugin/0.log"
Jul 18 02:44:09.750856 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:09.750828    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-s4hzq_59a3c995-844f-43c3-adc5-96c66692ecae/limitador/0.log"
Jul 18 02:44:10.565658 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:10.565625    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-grg8q_3a1dee76-44c1-4f6b-b4dc-0e947ce72231/kube-auth-proxy/0.log"
Jul 18 02:44:11.595838 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:11.595806    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8xvqb_2acfdadb-31e9-4fcd-93f7-2de584610d35/main/0.log"
Jul 18 02:44:11.709589 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:11.709562    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5wbn2_766eb2a3-9814-4993-81d7-c011d0824530/main/0.log"
Jul 18 02:44:11.935943 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:11.935823    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-442nf_dce77d3b-3729-4522-93c3-b233fab7c4ea/main/0.log"
Jul 18 02:44:20.549968 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:20.549934    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wpn87_b506e3e4-9db5-4d4f-86f4-88cca7974d24/global-pull-secret-syncer/0.log"
Jul 18 02:44:20.663074 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:20.663044    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fd7cq_294352e3-ce7b-4d1b-a742-1ed861819583/konnectivity-agent/0.log"
Jul 18 02:44:20.741698 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:20.741674    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-113.ec2.internal_a1303fa5083c2705ceafaaf4e41a3f21/haproxy/0.log"
Jul 18 02:44:26.304181 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:26.304151    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-w8h5t_9f61267b-b93b-45b4-8332-e0a31416924c/kuadrant-console-plugin/0.log"
Jul 18 02:44:26.537155 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:26.537130    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-s4hzq_59a3c995-844f-43c3-adc5-96c66692ecae/limitador/0.log"
Jul 18 02:44:28.288962 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.288931    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/alertmanager/0.log"
Jul 18 02:44:28.312933 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.312904    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/config-reloader/0.log"
Jul 18 02:44:28.336567 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.336546    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/kube-rbac-proxy-web/0.log"
Jul 18 02:44:28.360480 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.360459    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/kube-rbac-proxy/0.log"
Jul 18 02:44:28.381139 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.381119    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/kube-rbac-proxy-metric/0.log"
Jul 18 02:44:28.403339 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.403321    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/prom-label-proxy/0.log"
Jul 18 02:44:28.427183 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.427163    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7ecc2b0-91e8-44d6-804b-1542e88a4a6b/init-config-reloader/0.log"
Jul 18 02:44:28.499168 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.499144    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-4pvj8_acda8baf-5ef2-46bb-bb7a-90bed39b16ff/kube-state-metrics/0.log"
Jul 18 02:44:28.514577 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.514563    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-4pvj8_acda8baf-5ef2-46bb-bb7a-90bed39b16ff/kube-rbac-proxy-main/0.log"
Jul 18 02:44:28.536129 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.536111    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-4pvj8_acda8baf-5ef2-46bb-bb7a-90bed39b16ff/kube-rbac-proxy-self/0.log"
Jul 18 02:44:28.563609 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.563557    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-678c97459c-j4ctt_cdc861e9-c03e-4ae9-94e3-a16f4743df8f/metrics-server/0.log"
Jul 18 02:44:28.589645 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.589625    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-bhbzk_5ba1b558-ac73-4533-b25d-62e925dca452/monitoring-plugin/0.log"
Jul 18 02:44:28.619394 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.619373    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-55z2j_0fe0498d-6e61-41c2-add3-ee8a1db91da4/node-exporter/0.log"
Jul 18 02:44:28.637679 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.637663    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-55z2j_0fe0498d-6e61-41c2-add3-ee8a1db91da4/kube-rbac-proxy/0.log"
Jul 18 02:44:28.659627 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.659608    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-55z2j_0fe0498d-6e61-41c2-add3-ee8a1db91da4/init-textfile/0.log"
Jul 18 02:44:28.901593 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.901509    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5t5rk_350476df-fab6-46ec-ba57-27873edae6d7/kube-rbac-proxy-main/0.log"
Jul 18 02:44:28.922078 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.922055    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5t5rk_350476df-fab6-46ec-ba57-27873edae6d7/kube-rbac-proxy-self/0.log"
Jul 18 02:44:28.945185 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.945140    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5t5rk_350476df-fab6-46ec-ba57-27873edae6d7/openshift-state-metrics/0.log"
Jul 18 02:44:28.978124 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:28.978106    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/prometheus/0.log"
Jul 18 02:44:29.005129 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.005110    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/config-reloader/0.log"
Jul 18 02:44:29.030618 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.030602    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/thanos-sidecar/0.log"
Jul 18 02:44:29.057447 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.057428    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/kube-rbac-proxy-web/0.log"
Jul 18 02:44:29.083553 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.083532    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/kube-rbac-proxy/0.log"
Jul 18 02:44:29.104052 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.104031    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/kube-rbac-proxy-thanos/0.log"
Jul 18 02:44:29.134044 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.134008    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6d1a2a57-a2ab-40d4-a6f9-f6e030165821/init-config-reloader/0.log"
Jul 18 02:44:29.165899 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.165834    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-w44mp_05f9043f-2f88-4a2b-a45c-70eb7dc5136b/prometheus-operator/0.log"
Jul 18 02:44:29.182147 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.182129    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-w44mp_05f9043f-2f88-4a2b-a45c-70eb7dc5136b/kube-rbac-proxy/0.log"
Jul 18 02:44:29.207601 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.207578    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-7tvqs_15b3b6fe-a4ac-4887-a8b1-7e5c6f2b1022/prometheus-operator-admission-webhook/0.log"
Jul 18 02:44:29.332867 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.332847    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-699ff677f6-mk99f_39033c8c-819c-412c-a159-f81ab0e3b072/thanos-query/0.log"
Jul 18 02:44:29.353994 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.353975    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-699ff677f6-mk99f_39033c8c-819c-412c-a159-f81ab0e3b072/kube-rbac-proxy-web/0.log"
Jul 18 02:44:29.372785 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.372764    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-699ff677f6-mk99f_39033c8c-819c-412c-a159-f81ab0e3b072/kube-rbac-proxy/0.log"
Jul 18 02:44:29.393011 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.392989    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-699ff677f6-mk99f_39033c8c-819c-412c-a159-f81ab0e3b072/prom-label-proxy/0.log"
Jul 18 02:44:29.412761 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.412743    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-699ff677f6-mk99f_39033c8c-819c-412c-a159-f81ab0e3b072/kube-rbac-proxy-rules/0.log"
Jul 18 02:44:29.433649 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:29.433596    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-699ff677f6-mk99f_39033c8c-819c-412c-a159-f81ab0e3b072/kube-rbac-proxy-metrics/0.log"
Jul 18 02:44:30.340703 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.340670    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"]
Jul 18 02:44:30.341082 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.341028    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9b52e2-329c-4441-89cf-3c59adc4ee8c" containerName="revoke-keys"
Jul 18 02:44:30.341082 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.341040    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9b52e2-329c-4441-89cf-3c59adc4ee8c" containerName="revoke-keys"
Jul 18 02:44:30.341223 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.341106    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db9b52e2-329c-4441-89cf-3c59adc4ee8c" containerName="revoke-keys"
Jul 18 02:44:30.344312 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.344293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.346715 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.346693    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vl8f\"/\"kube-root-ca.crt\""
Jul 18 02:44:30.346849 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.346693    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vl8f\"/\"openshift-service-ca.crt\""
Jul 18 02:44:30.346849 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.346824    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6vl8f\"/\"default-dockercfg-hrtqd\""
Jul 18 02:44:30.354815 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.354795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"]
Jul 18 02:44:30.429469 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.429442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-podres\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.429469 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.429470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzzhp\" (UniqueName: \"kubernetes.io/projected/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-kube-api-access-rzzhp\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.429634 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.429494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-lib-modules\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.429634 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.429589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-sys\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.429702 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.429647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-proc\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530034 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-proc\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530152 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-podres\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530152 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzzhp\" (UniqueName: \"kubernetes.io/projected/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-kube-api-access-rzzhp\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530152 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-lib-modules\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530152 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-proc\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530152 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-sys\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530378 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530218    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-sys\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530378 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-podres\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.530378 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.530301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-lib-modules\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.538287 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.538269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzzhp\" (UniqueName: \"kubernetes.io/projected/b82630b3-b2dd-4aef-8504-7bb9cbfb86b1-kube-api-access-rzzhp\") pod \"perf-node-gather-daemonset-twknp\" (UID: \"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1\") " pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.654839 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.654779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:30.819405 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:30.819381    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"]
Jul 18 02:44:30.821658 ip-10-0-133-113 kubenswrapper[2576]: W0718 02:44:30.821629    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb82630b3_b2dd_4aef_8504_7bb9cbfb86b1.slice/crio-fe5aa98366744eaff1f920b3043897834726b5badcb79551c2d95ac9d832e9e9 WatchSource:0}: Error finding container fe5aa98366744eaff1f920b3043897834726b5badcb79551c2d95ac9d832e9e9: Status 404 returned error can't find the container with id fe5aa98366744eaff1f920b3043897834726b5badcb79551c2d95ac9d832e9e9
Jul 18 02:44:31.611358 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:31.611327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp" event={"ID":"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1","Type":"ContainerStarted","Data":"b68f2ab69a4f7ab4cc46cf9934824dc9599345db82210cedab2a116426cf1920"}
Jul 18 02:44:31.611766 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:31.611388    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:31.611766 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:31.611411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp" event={"ID":"b82630b3-b2dd-4aef-8504-7bb9cbfb86b1","Type":"ContainerStarted","Data":"fe5aa98366744eaff1f920b3043897834726b5badcb79551c2d95ac9d832e9e9"}
Jul 18 02:44:31.630101 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:31.630035    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp" podStartSLOduration=1.630019369 podStartE2EDuration="1.630019369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:44:31.628913285 +0000 UTC m=+2846.121557112" watchObservedRunningTime="2026-07-18 02:44:31.630019369 +0000 UTC m=+2846.122663195"
Jul 18 02:44:33.325759 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:33.325718    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6r2lr_181cb719-070d-4ea8-800c-c7b54a6a915b/dns/0.log"
Jul 18 02:44:33.349503 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:33.349478    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6r2lr_181cb719-070d-4ea8-800c-c7b54a6a915b/kube-rbac-proxy/0.log"
Jul 18 02:44:33.536410 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:33.536384    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ml5cr_91dea430-55c6-42e2-9589-148379a893aa/dns-node-resolver/0.log"
Jul 18 02:44:34.125989 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:34.125960    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdb4x_1d780625-cebc-417b-9f06-1f00ec4f972b/node-ca/0.log"
Jul 18 02:44:35.207425 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:35.207398    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-grg8q_3a1dee76-44c1-4f6b-b4dc-0e947ce72231/kube-auth-proxy/0.log"
Jul 18 02:44:36.076209 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:36.076152    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mtpxd_dfc9fb34-b2f3-467a-ae70-77afade7a241/serve-healthcheck-canary/0.log"
Jul 18 02:44:36.581838 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:36.581812    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nktd6_7f0d691e-e9f8-484e-ad9b-00acdd3bf10c/kube-rbac-proxy/0.log"
Jul 18 02:44:36.600332 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:36.600312    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nktd6_7f0d691e-e9f8-484e-ad9b-00acdd3bf10c/exporter/0.log"
Jul 18 02:44:36.621599 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:36.621578    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nktd6_7f0d691e-e9f8-484e-ad9b-00acdd3bf10c/extractor/0.log"
Jul 18 02:44:37.626037 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:37.626010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6vl8f/perf-node-gather-daemonset-twknp"
Jul 18 02:44:39.560137 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:39.560062    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-rczg9_7cb54667-9934-49a0-8753-8c2d8f5abc15/manager/0.log"
Jul 18 02:44:39.579341 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:39.579316    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d87dd68c9-rczg9_7cb54667-9934-49a0-8753-8c2d8f5abc15/copy-manifests/0.log"
Jul 18 02:44:40.713726 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:40.713700    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-grg8q_3a1dee76-44c1-4f6b-b4dc-0e947ce72231/kube-auth-proxy/0.log"
Jul 18 02:44:40.892702 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:40.892669    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b64b4b7d4-p6bqz_88b9bb0d-897f-4b9d-88fb-9872e307d87e/manager/0.log"
Jul 18 02:44:42.181377 ip-10-0-133-113 kubenswrapper[2576]: I0718 02:44:42.181298    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-748465cbd5-grg8q_3a1dee76-44c1-4f6b-b4dc-0e947ce72231/kube-auth-proxy/0.log"