Jul 06 23:54:14.507868 ip-10-0-141-68 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 23:54:14.507877 ip-10-0-141-68 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 23:54:14.507884 ip-10-0-141-68 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 23:54:14.508216 ip-10-0-141-68 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 23:54:24.164781 ip-10-0-141-68 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 23:54:24.164793 ip-10-0-141-68 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6c80b6dead834d13976a43f3b40e4d1e --
Jul 06 23:56:15.511913 ip-10-0-141-68 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 23:56:15.927204 ip-10-0-141-68 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:15.927204 ip-10-0-141-68 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 23:56:15.927204 ip-10-0-141-68 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:15.927204 ip-10-0-141-68 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 23:56:15.927204 ip-10-0-141-68 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:15.928509 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.928162    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 23:56:15.932974 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932948    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:15.932974 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932972    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:15.932974 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932976    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:15.932974 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932978    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932982    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932985    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932990    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932992    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932995    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.932998    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:15.933130 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933064    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933222    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933229    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933234    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933241    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933246    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933251    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933256    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933261    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933265    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933270    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933274    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933282    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933293    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933297    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933301    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933306    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933310    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933314    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:15.933404 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933318    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933322    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933327    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933330    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933334    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933338    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933342    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933351    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933355    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933359    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933368    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933374    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933378    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933383    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933399    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933404    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933408    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933413    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933418    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:15.934133 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933428    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933432    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933436    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933440    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933444    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933448    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933453    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933457    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933461    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933465    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933469    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933473    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933482    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933486    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933491    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933495    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933499    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933545    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933552    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:15.934764 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933557    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933561    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933566    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933570    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933574    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933584    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933588    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933592    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933596    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933603    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933608    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933612    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933617    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933621    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933625    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933629    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933633    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933637    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933645    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933650    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:15.935237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933654    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933659    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933663    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933667    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933671    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933676    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933680    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933684    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933688    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933693    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933697    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.933708    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935158    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935170    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935173    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935177    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935180    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935183    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935186    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:15.935761 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935189    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935192    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935195    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935198    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935201    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935203    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935206    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935209    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935211    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935214    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935216    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935219    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935221    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935224    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935227    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935230    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935232    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935235    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935237    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935240    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:15.936214 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935243    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935246    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935248    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935251    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935254    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935256    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935262    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935264    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935267    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935269    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935272    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935274    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935277    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935280    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935285    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935289    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935292    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935295    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935298    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935301    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:15.936748 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935304    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935306    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935309    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935312    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935315    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935317    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935319    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935322    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935324    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935327    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935330    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935332    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935335    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935338    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935341    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935343    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935346    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935351    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935356    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935359    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:15.937243 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935362    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935365    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935379    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935384    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935386    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935389    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935392    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935401    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935404    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935406    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935409    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935412    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935414    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935417    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935420    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935424    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935427    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935430    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935432    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:15.937747 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935435    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935437    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935440    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935442    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935444    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935447    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935449    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935452    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935459    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935462    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935464    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935469    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935471    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935927    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935933    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935936    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935939    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935941    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935944    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935947    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:15.938237 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935949    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935952    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935954    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935957    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935959    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935964    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935967    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935970    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935973    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935976    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935978    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935981    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935984    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935986    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935989    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935991    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935994    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935996    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.935999    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936002    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:15.938792 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936004    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936007    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936009    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936014    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936017    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936019    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936022    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936024    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936027    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936030    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936032    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936034    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936037    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936040    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936042    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936044    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936047    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936049    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936052    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936054    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:15.939289 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936057    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936059    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936062    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936064    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936066    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936069    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936071    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936073    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936075    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936078    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936082    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936085    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936088    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936091    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936094    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936098    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936101    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936104    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936106    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:15.939845 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936109    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936112    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936115    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936117    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936120    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936122    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936124    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936127    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936129    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936132    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936134    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936137    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936140    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936142    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936144    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936147    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936150    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936152    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936154    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:15.940316 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936157    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936159    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936161    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936164    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936167    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936170    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936173    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936176    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936178    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936182    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936184    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936187    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936189    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936192    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936281    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936290    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936298    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936303    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936307    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936310    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936314    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 23:56:15.940837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936319    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936323    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936328    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936334    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936339    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936344    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936348    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936352    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936356    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936361    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936366    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936370    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936373    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936376    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936379    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936383    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936387    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936390    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936393    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936397    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936402    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936405    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936408    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936411    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 23:56:15.941352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936414    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936418    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936421    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936424    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936427    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936430    2583 flags.go:64] FLAG: --enable-server="true"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936433    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936441    2583 flags.go:64] FLAG: --event-burst="100"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936444    2583 flags.go:64] FLAG: --event-qps="50"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936447    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936450    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936453    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936457    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936460    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936463    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936466    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936468    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936471    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936474    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936477    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936479    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936484    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936487    2583 flags.go:64] FLAG: --feature-gates=""
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936490    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936493    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 23:56:15.941944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936496    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936499    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936502    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936507    2583 flags.go:64] FLAG: --help="false"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936510    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-68.ec2.internal"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936513    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936516    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936518    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936539    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936544    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936547    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936550    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936553    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936555    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936558    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936562    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936564    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936567    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936570    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936573    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936575    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936578    2583 flags.go:64] FLAG: --lock-file=""
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936581    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936583    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 23:56:15.942595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936586    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936592    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936594    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936597    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936601    2583 flags.go:64] FLAG: --logging-format="text"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936604    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936607    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936610    2583 flags.go:64] FLAG: --manifest-url=""
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936613    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936620    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936623    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936629    2583 flags.go:64] FLAG: --max-pods="110"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936633    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936636    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936639    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936641    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936644    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936647    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936650    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936658    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936661    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936664    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936667    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 06 23:56:15.943229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936669    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936675    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936678    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936681    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936683    2583 flags.go:64] FLAG: --port="10250"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936686    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936689    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0317201498d3ff5b6"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936692    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936695    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936698    2583 flags.go:64] FLAG: --register-node="true"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936701    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936704    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936707    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936710    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936715    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936719    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936722    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936725    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936728    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936731    2583 flags.go:64] FLAG: --runonce="false"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936735    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936738    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936742    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936745    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936748    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936751    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 23:56:15.943858 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936755    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936758    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936760    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936763    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936766    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936769    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936772    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936775    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936778    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936782    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936785    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936788    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936791    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936794    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936797    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936799    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936802    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936805    2583 flags.go:64] FLAG: --v="2"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936809    2583 flags.go:64] FLAG: --version="false"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936813    2583 flags.go:64] FLAG: --vmodule=""
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936818    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.936821    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936971    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936975    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936978    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:15.944508 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936982    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936986    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936989    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936991    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936994    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936997    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.936999    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937002    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937004    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937007    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937009    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937012    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937014    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937016    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937019    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937022    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937024    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937027    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937029    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:15.945137 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937032    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937034    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937036    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937039    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937041    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937044    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937046    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937049    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937053    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937055    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937058    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937060    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937064    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937068    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937071    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937075    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937077    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937080    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937083    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:15.945627 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937086    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937089    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937091    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937094    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937096    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937099    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937101    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937103    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937106    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937109    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937111    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937113    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937116    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937118    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937121    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937123    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937126    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937129    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937133    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:15.946100 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937136    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937139    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937144    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937146    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937149    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937152    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937154    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937157    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937159    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937162    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937164    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937167    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937169    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937172    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937175    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937177    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937180    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937182    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937185    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937187    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:15.946579 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937189    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937192    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937194    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937197    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937199    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937202    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937204    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937207    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937209    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937211    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937214    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937216    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937219    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937222    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937226    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937229    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937232    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937235    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:15.947068 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.937237    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.937782    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945027    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945048    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945070    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945077    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945138    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945145    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945150    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945154    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945157    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945159    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945162    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945165    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:15.947511 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945168    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945171    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945173    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945176    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945179    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945182    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945185    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945188    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945190    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945193    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945195    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945198    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945200    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945202    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945205    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945207    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945210    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945212    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945215    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945217    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:15.947887 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945221    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945225    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945227    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945230    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945232    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945235    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945237    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945240    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945242    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945245    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945247    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945249    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945252    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945255    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945258    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945260    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945262    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945265    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945267    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945270    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:15.948446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945272    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945275    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945277    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945279    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945282    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945285    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945288    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945292    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945295    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945298    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945301    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945304    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945307    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945311    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945314    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945317    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945320    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945324    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945329    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:15.948991 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945332    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945335    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945338    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945340    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945343    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945346    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945348    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945351    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945354    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945356    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945359    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945361    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945363    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945366    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945368    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945371    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945373    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945375    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945378    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945380    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:15.949464 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945383    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945385    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945388    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945390    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945393    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945396    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945399    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945401    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945405    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945407    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945410    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945412    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945417    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945557    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945564    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:15.950065 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945567    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945570    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945573    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945575    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945578    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945581    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945583    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945586    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945589    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945591    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945593    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945596    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945599    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945601    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945603    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945606    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945608    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945611    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945613    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945615    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:15.950446 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945618    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945621    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945624    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945628    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945632    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945635    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945640    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945643    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945646    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945648    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945651    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945654    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945657    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945660    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945663    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945667    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945670    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945673    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945675    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:15.950998 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945678    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945680    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945683    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945686    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945688    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945691    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945694    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945696    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945699    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945701    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945704    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945706    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945708    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945711    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945713    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945715    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945718    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945720    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945723    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945725    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:15.951649 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945729    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945731    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945734    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945736    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945739    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945741    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945744    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945746    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945748    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945750    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945753    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945755    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945757    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945760    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945762    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945765    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945767    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945770    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945772    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945775    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:15.952197 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945778    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945780    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945783    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945785    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945788    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945791    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945794    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945797    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945799    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945802    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945804    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945807    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945809    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945812    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945814    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945817    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945819    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:15.952865 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:15.945822    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945827    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.945966    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.948068    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.949509    2583 server.go:1019] "Starting client certificate rotation"
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.949638    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.949700    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.952114    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:15.953491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.953296    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:15.963121 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.963099    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 06 23:56:15.966456 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.966433    2583 log.go:25] "Validated CRI v1 image API"
Jul 06 23:56:15.967963 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.967947    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 23:56:15.970538 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.970495    2583 fs.go:135] Filesystem UUIDs: map[12060760-10ee-4d50-80d2-2791670300c5:/dev/nvme0n1p3 76d5b54d-bb4f-4ba0-a44f-dfcfdda65757:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 06 23:56:15.970602 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.970538    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 23:56:15.975785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.975650    2583 manager.go:216] Machine: {Timestamp:2026-07-06 23:56:15.974289443 +0000 UTC m=+0.339593181 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105547 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22a288e4fe89e709969ce27205d923 SystemUUID:ec22a288-e4fe-89e7-0996-9ce27205d923 BootID:6c80b6de-ad83-4d13-976a-43f3b40e4d1e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:4e:1e:02:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:4e:1e:02:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:55:2e:3d:26:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 23:56:15.975785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.975774    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 23:56:15.975904 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.975868    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 23:56:15.979006 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.978966    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 23:56:15.979206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.979011    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 23:56:15.979843 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.979829    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 23:56:15.979909 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.979849    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 23:56:15.979909 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.979861    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:15.980250 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.980231    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:15.980476 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.980460    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:15.981793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.981779    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 23:56:15.984729 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984711    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:15.984891 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984879    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 23:56:15.984962 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984923    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 23:56:15.984962 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984939    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 23:56:15.984962 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984960    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 23:56:15.985089 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984976    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 06 23:56:15.985089 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.984990    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 23:56:15.985443 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.985427    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:15.985509 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.985450    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:15.987867 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.987841    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 23:56:15.989952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.989937    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 23:56:15.990003 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.989970    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 23:56:15.991086 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991074    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 23:56:15.991086 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991090    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991096    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991102    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991110    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991115    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991123    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991128    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991136    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991141    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991151    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 23:56:15.991190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991156    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 23:56:15.991908 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991897    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 23:56:15.991908 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.991907    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 23:56:15.995384 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.995360    2583 server.go:1268] "Started kubelet"
Jul 06 23:56:15.995609 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.995548    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 23:56:15.995704 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.995630    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 23:56:15.995704 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.995672    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 23:56:15.995845 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.995575    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 23:56:15.996355 ip-10-0-141-68 systemd[1]: Started Kubernetes Kubelet.
Jul 06 23:56:15.996937 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:15.996800    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 23:56:15.996937 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:15.996797    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 23:56:15.997758 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.997740    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 23:56:15.999872 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:15.999856    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 23:56:16.002105 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.002078    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9crbd"
Jul 06 23:56:16.003018 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.003002    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 23:56:16.004358 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.002412    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-68.ec2.internal.18bfd8a3eaf6f0e8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-68.ec2.internal,UID:ip-10-0-141-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-68.ec2.internal,},FirstTimestamp:2026-07-06 23:56:15.995326696 +0000 UTC m=+0.360630433,LastTimestamp:2026-07-06 23:56:15.995326696 +0000 UTC m=+0.360630433,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-68.ec2.internal,}"
Jul 06 23:56:16.008797 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.008776    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9crbd"
Jul 06 23:56:16.009944 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.009930    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 23:56:16.010022 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.009969    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:16.010149 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.010128    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.010709 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.010642    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 23:56:16.010783 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.010737    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 23:56:16.010783 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.010753    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 23:56:16.010927 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.010904    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 23:56:16.010927 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.010915    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 23:56:16.011198 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011075    2583 factory.go:55] Registering systemd factory
Jul 06 23:56:16.011198 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011088    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 06 23:56:16.011359 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011345    2583 factory.go:153] Registering CRI-O factory
Jul 06 23:56:16.011359 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011360    2583 factory.go:223] Registration of the crio container factory successfully
Jul 06 23:56:16.011494 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011429    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 23:56:16.011494 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011450    2583 factory.go:103] Registering Raw factory
Jul 06 23:56:16.011494 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.011461    2583 manager.go:1196] Started watching for new ooms in manager
Jul 06 23:56:16.012164 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.012144    2583 manager.go:349] Starting recovery of all containers
Jul 06 23:56:16.012761 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.012732    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 23:56:16.012862 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.012847    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 23:56:16.021796 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.021663    2583 manager.go:354] Recovery completed
Jul 06 23:56:16.026141 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.026123    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.028251 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028229    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.028336 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028266    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.028336 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028277    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.028932 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028902    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 23:56:16.028932 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028917    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 23:56:16.028932 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028928    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.029102 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.028936    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:16.030038 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.030022    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.030119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.030053    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.030119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.030062    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.030239 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.030146    2583 policy_none.go:49] "None policy: Start"
Jul 06 23:56:16.030239 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.030158    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 23:56:16.030239 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.030168    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 23:56:16.031628 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.031615    2583 policy_none.go:47] "Start"
Jul 06 23:56:16.069267 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069244    2583 manager.go:337] "Starting Device Plugin manager"
Jul 06 23:56:16.069391 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.069283    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 23:56:16.069391 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069298    2583 server.go:87] "Starting device plugin registration server"
Jul 06 23:56:16.069645 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069632    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 23:56:16.069692 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069646    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 23:56:16.069789 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069773    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 23:56:16.069878 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069860    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 23:56:16.069923 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.069881    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 23:56:16.070496 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.070474    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 23:56:16.070606 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.070561    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.149477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.149431    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 23:56:16.150869 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.150851    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 23:56:16.150930 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.150894    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 23:56:16.150930 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.150923    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 23:56:16.151013 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.150969    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 23:56:16.153223 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.153197    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:16.170132 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.170092    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.171177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.171158    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.171285 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.171192    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.171285 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.171203    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.171285 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.171227    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.177557 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.177487    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.177557 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.177514    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-68.ec2.internal\": node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.196660 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.196625    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.251297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.251257    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"]
Jul 06 23:56:16.251475 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.251339    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.253166 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.253150    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.253238 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.253181    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.253238 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.253191    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.254685 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.254672    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.254804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.254789    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.254854 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.254820    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.255457 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.255439    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.255559 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.255473    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.255559 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.255483    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.255559 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.255439    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.255661 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.255563    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.255661 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.255577    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.256837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.256820    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.256894 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.256857    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:16.257623 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.257606    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:16.257706 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.257637    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:16.257706 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.257649    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:16.279455 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.279427    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-68.ec2.internal\" not found" node="ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.283077 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.283058    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-68.ec2.internal\" not found" node="ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.297705 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.297677    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.313888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.313861    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9d0657b49ada5194a7ba0bead0569b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal\" (UID: \"d9d0657b49ada5194a7ba0bead0569b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.313985 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.313891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a09046d52a93c64b9a467e152c6a69f0-config\") pod \"kube-apiserver-proxy-ip-10-0-141-68.ec2.internal\" (UID: \"a09046d52a93c64b9a467e152c6a69f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.313985 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.313909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d9d0657b49ada5194a7ba0bead0569b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal\" (UID: \"d9d0657b49ada5194a7ba0bead0569b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.398802 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.398760    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.414680 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.414643    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9d0657b49ada5194a7ba0bead0569b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal\" (UID: \"d9d0657b49ada5194a7ba0bead0569b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.414781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.414688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a09046d52a93c64b9a467e152c6a69f0-config\") pod \"kube-apiserver-proxy-ip-10-0-141-68.ec2.internal\" (UID: \"a09046d52a93c64b9a467e152c6a69f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.414781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.414716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d9d0657b49ada5194a7ba0bead0569b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal\" (UID: \"d9d0657b49ada5194a7ba0bead0569b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.414781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.414748    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9d0657b49ada5194a7ba0bead0569b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal\" (UID: \"d9d0657b49ada5194a7ba0bead0569b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.414781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.414778    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d9d0657b49ada5194a7ba0bead0569b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal\" (UID: \"d9d0657b49ada5194a7ba0bead0569b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.414936 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.414755    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a09046d52a93c64b9a467e152c6a69f0-config\") pod \"kube-apiserver-proxy-ip-10-0-141-68.ec2.internal\" (UID: \"a09046d52a93c64b9a467e152c6a69f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.500034 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.499943    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.587107 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.587053    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.587198 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.587053    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:16.600504 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.600466    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.701176 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.701123    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.803748 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.803638    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.904754 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:16.904711    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:16.949803 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.949749    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 23:56:16.950387 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:16.949972    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:56:17.005626 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:17.005587    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:17.014799 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.014762    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:51:16 +0000 UTC" deadline="2028-01-04 10:10:36.277610581 +0000 UTC"
Jul 06 23:56:17.014799 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.014798    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13114h14m19.26281622s"
Jul 06 23:56:17.106238 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:17.106183    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:17.108704 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.108677    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:17.139225 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.139196    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:17.173129 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:17.173076    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09046d52a93c64b9a467e152c6a69f0.slice/crio-7297647f423382abca4567a70819444e99fb76f539cd50c01a2c2f689a6a32de WatchSource:0}: Error finding container 7297647f423382abca4567a70819444e99fb76f539cd50c01a2c2f689a6a32de: Status 404 returned error can't find the container with id 7297647f423382abca4567a70819444e99fb76f539cd50c01a2c2f689a6a32de
Jul 06 23:56:17.173405 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:17.173386    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d0657b49ada5194a7ba0bead0569b2.slice/crio-01d223b44caefecb7b9060f899e828387eec0f98e21c74facc71392a3d3877ce WatchSource:0}: Error finding container 01d223b44caefecb7b9060f899e828387eec0f98e21c74facc71392a3d3877ce: Status 404 returned error can't find the container with id 01d223b44caefecb7b9060f899e828387eec0f98e21c74facc71392a3d3877ce
Jul 06 23:56:17.177933 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.177911    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:56:17.207255 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:17.207213    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:17.313836 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:17.313796    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-68.ec2.internal\" not found"
Jul 06 23:56:17.363275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.363183    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:17.411343 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.411305    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:17.423354 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.423321    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:17.424367 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.424350    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal"
Jul 06 23:56:17.431624 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.431592    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:17.988551 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.988496    2583 apiserver.go:52] "Watching apiserver"
Jul 06 23:56:17.998116 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.998079    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 23:56:17.999422 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:17.999390    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-w272k","openshift-multus/multus-additional-cni-plugins-96jnk","openshift-multus/network-metrics-daemon-fwwp7","openshift-network-diagnostics/network-check-target-6wfrf","kube-system/global-pull-secret-syncer-zkcfs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal","openshift-multus/multus-9sp9k","openshift-network-operator/iptables-alerter-9kprk","openshift-ovn-kubernetes/ovnkube-node-zdsfs","kube-system/konnectivity-agent-wqkxt","kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jbv5t","openshift-dns/node-resolver-xr9cd"]
Jul 06 23:56:18.001419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.001391    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.003185 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.003164    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.004589 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.004567    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.004715 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.004649    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dwxvx\""
Jul 06 23:56:18.004715 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.004646    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:18.005065 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.005033    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.005184 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.005165    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.005276 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.005183    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 23:56:18.005341 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.005294    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 23:56:18.005644 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.005623    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 23:56:18.005717 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.005700    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 23:56:18.006416 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.006382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:18.006557 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.006531    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:18.008159 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.008138    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.008682 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.008651    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dsjpt\""
Jul 06 23:56:18.008942 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.008922    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.009298 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.009274    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.009411 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.009381    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:18.011042 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.011017    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.012236 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.011504    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:18.015707 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.013684    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.015707 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.014999    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gkwdr\""
Jul 06 23:56:18.015707 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.015159    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.015707 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.015299    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.016368 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.015792    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 23:56:18.016368 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.015995    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.016915 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.016630    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 23:56:18.016915 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.016658    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.016915 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.016725    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hcl9l\""
Jul 06 23:56:18.016915 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.016898    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.017229 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.016931    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.018035 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.018014    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.019180 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.018423    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.019180 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.018704    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.019180 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.018729    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bz4s5\""
Jul 06 23:56:18.019359 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.019340    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.019681 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.019662    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.019768 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.019726    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vjxhh\""
Jul 06 23:56:18.019850 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.019664    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 23:56:18.019916 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.019853    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 23:56:18.019982 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.019959    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 23:56:18.020334 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.020317    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.020952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.020637    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 23:56:18.020952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.020936    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 23:56:18.021154 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.020936    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-llfkr\""
Jul 06 23:56:18.021154 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.020936    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 23:56:18.021959 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.021938    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.022702 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.022680    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xxfhz\""
Jul 06 23:56:18.023257 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.023080    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 23:56:18.023257 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.023099    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 23:56:18.023997 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.023974    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 23:56:18.024273 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024252    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bsc6f\""
Jul 06 23:56:18.024666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024644    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-lib-modules\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.024753 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-host\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.024753 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4n8w\" (UniqueName: \"kubernetes.io/projected/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-kube-api-access-t4n8w\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.024861 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.024861 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024780    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-tuned\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.024861 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024796    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-iptables-alerter-script\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.024861 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024827    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-host-slash\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.024861 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-registration-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024883    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whlvl\" (UniqueName: \"kubernetes.io/projected/60418f11-dff3-480e-a1f3-344f9033d62a-kube-api-access-whlvl\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024911    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-var-lib-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b889e54-05f4-424c-8db8-9b28dc7e15c7-tmp\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024945    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6ct2\" (UniqueName: \"kubernetes.io/projected/5b889e54-05f4-424c-8db8-9b28dc7e15c7-kube-api-access-b6ct2\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.024963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-kubelet\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025006    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-run-netns\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025026    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-systemd\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-node-log\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025059    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-ovnkube-script-lib\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025090 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysconfig\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025100    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-kubernetes\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025115    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-cni-netd\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-ovnkube-config\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025161    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-run\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-slash\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da53e071-0353-4538-8837-240274613bc2-kubelet-config\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025247    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbxz7\" (UniqueName: \"kubernetes.io/projected/448d3e76-6d59-46b3-af7d-7714f0fd3772-kube-api-access-nbxz7\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025283    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-sys-fs\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025315    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-systemd-units\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025337    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da53e071-0353-4538-8837-240274613bc2-dbus\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025369    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:18.025382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-device-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025402    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-etc-selinux\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025426    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-log-socket\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-run-ovn-kubernetes\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-socket-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-cni-bin\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-env-overrides\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025537    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2cx\" (UniqueName: \"kubernetes.io/projected/36d5608a-97e8-4e15-8a53-bb223736daeb-kube-api-access-hk2cx\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-modprobe-d\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-sys\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025647    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-var-lib-kubelet\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-etc-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-ovn\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/36d5608a-97e8-4e15-8a53-bb223736daeb-ovn-node-metrics-cert\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysctl-d\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.025860 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysctl-conf\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.026441 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-systemd\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.026441 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.025987    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:18.050423 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.050394    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cdchc"
Jul 06 23:56:18.059450 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.059416    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cdchc"
Jul 06 23:56:18.112455 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.112421    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 23:56:18.126590 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126551    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4n8w\" (UniqueName: \"kubernetes.io/projected/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-kube-api-access-t4n8w\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.126785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126597    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.126785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126627    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-system-cni-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.126785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-cnibin\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.126785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-iptables-alerter-script\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.126785 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126718    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-registration-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whlvl\" (UniqueName: \"kubernetes.io/projected/60418f11-dff3-480e-a1f3-344f9033d62a-kube-api-access-whlvl\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126933    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6ct2\" (UniqueName: \"kubernetes.io/projected/5b889e54-05f4-424c-8db8-9b28dc7e15c7-kube-api-access-b6ct2\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbhpx\" (UniqueName: \"kubernetes.io/projected/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-kube-api-access-jbhpx\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.126995    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f-konnectivity-ca\") pod \"konnectivity-agent-wqkxt\" (UID: \"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f\") " pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127001    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-registration-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-socket-dir-parent\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.127063 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vglh8\" (UniqueName: \"kubernetes.io/projected/b8e53e88-0804-4c75-b30f-263da33307d0-kube-api-access-vglh8\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-kubelet\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127109    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-run-netns\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysconfig\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-kubernetes\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-cni-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127212    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae566a7-243e-46e2-9bd7-e58e689eb474-tmp-dir\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-multus-certs\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-slash\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127307    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-iptables-alerter-script\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-sys-fs\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-systemd-units\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-run-ovn-kubernetes\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-systemd\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127426    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysconfig\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:18.127542 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-etc-selinux\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127382    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-run-netns\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127516    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-kubelet\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127570    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-sys-fs\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127608    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-kubernetes\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127644    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-systemd-units\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127658    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-systemd\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127674    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-etc-selinux\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127684    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-slash\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-log-socket\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127713    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-run-ovn-kubernetes\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-cni-bin\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127768    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-log-socket\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127779    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-cni-bin\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127788    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-env-overrides\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2cx\" (UniqueName: \"kubernetes.io/projected/36d5608a-97e8-4e15-8a53-bb223736daeb-kube-api-access-hk2cx\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127865    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70b6c117-9122-43d1-aa4f-bee759d72e46-host\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.127900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f-agent-certs\") pod \"konnectivity-agent-wqkxt\" (UID: \"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f\") " pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.128419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b889e54-05f4-424c-8db8-9b28dc7e15c7-tmp\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bae566a7-243e-46e2-9bd7-e58e689eb474-hosts-file\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-socket-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-modprobe-d\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-sys\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-var-lib-kubelet\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128257    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128278    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-env-overrides\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128286    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-socket-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128287    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-etc-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128333    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-ovn\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/36d5608a-97e8-4e15-8a53-bb223736daeb-ovn-node-metrics-cert\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysctl-d\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysctl-conf\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128396    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128427    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128500    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-host\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.129194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128353    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-sys\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128540    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-modprobe-d\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128644    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysctl-d\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128697    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-sysctl-conf\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128711    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-etc-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128713    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-var-lib-kubelet\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128745    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-ovn\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128450    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-host\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-tuned\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-system-cni-dir\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128916    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znk8r\" (UniqueName: \"kubernetes.io/projected/70b6c117-9122-43d1-aa4f-bee759d72e46-kube-api-access-znk8r\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-os-release\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-hostroot\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.128988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-conf-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-host-slash\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-var-lib-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129081    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cnibin\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.130224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129107    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-cni-multus\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-systemd\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-node-log\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129213    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-ovnkube-script-lib\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/70b6c117-9122-43d1-aa4f-bee759d72e46-serviceca\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8e53e88-0804-4c75-b30f-263da33307d0-cni-binary-copy\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129305    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-netns\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8e53e88-0804-4c75-b30f-263da33307d0-multus-daemon-config\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-cni-netd\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-ovnkube-config\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-run\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da53e071-0353-4538-8837-240274613bc2-kubelet-config\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129474    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-node-log\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129529    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-host-cni-netd\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129610    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-host-slash\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129658    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-var-lib-openvswitch\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.131171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129741    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/36d5608a-97e8-4e15-8a53-bb223736daeb-run-systemd\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129810    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbxz7\" (UniqueName: \"kubernetes.io/projected/448d3e76-6d59-46b3-af7d-7714f0fd3772-kube-api-access-nbxz7\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129861    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-lib-modules\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129888    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-kubelet\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da53e071-0353-4538-8837-240274613bc2-dbus\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.129983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130011    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-device-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130038    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-os-release\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130059    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-ovnkube-config\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-k8s-cni-cncf-io\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130121    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-run\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.130211    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.130345    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:18.630259331 +0000 UTC m=+2.995563056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130666    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/36d5608a-97e8-4e15-8a53-bb223736daeb-ovnkube-script-lib\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-cni-bin\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.137297 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130741    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-etc-kubernetes\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bkd5\" (UniqueName: \"kubernetes.io/projected/bae566a7-243e-46e2-9bd7-e58e689eb474-kube-api-access-9bkd5\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130870    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130873    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da53e071-0353-4538-8837-240274613bc2-dbus\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130123    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da53e071-0353-4538-8837-240274613bc2-kubelet-config\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.130925    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.130938    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.130998    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:18.630980813 +0000 UTC m=+2.996284554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.131014    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/60418f11-dff3-480e-a1f3-344f9033d62a-device-dir\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.131099    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b889e54-05f4-424c-8db8-9b28dc7e15c7-lib-modules\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.131921    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b889e54-05f4-424c-8db8-9b28dc7e15c7-tmp\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.131969    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b889e54-05f4-424c-8db8-9b28dc7e15c7-etc-tuned\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.138268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.132116    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/36d5608a-97e8-4e15-8a53-bb223736daeb-ovn-node-metrics-cert\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.146363 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.146278    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:18.146363 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.146314    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:18.146363 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.146330    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:18.146652 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.146416    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:18.646395404 +0000 UTC m=+3.011699135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:18.148008 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.147973    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whlvl\" (UniqueName: \"kubernetes.io/projected/60418f11-dff3-480e-a1f3-344f9033d62a-kube-api-access-whlvl\") pod \"aws-ebs-csi-driver-node-4mjcm\" (UID: \"60418f11-dff3-480e-a1f3-344f9033d62a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.150392 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.150254    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6ct2\" (UniqueName: \"kubernetes.io/projected/5b889e54-05f4-424c-8db8-9b28dc7e15c7-kube-api-access-b6ct2\") pod \"tuned-jbv5t\" (UID: \"5b889e54-05f4-424c-8db8-9b28dc7e15c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.150392 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.150330    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2cx\" (UniqueName: \"kubernetes.io/projected/36d5608a-97e8-4e15-8a53-bb223736daeb-kube-api-access-hk2cx\") pod \"ovnkube-node-zdsfs\" (UID: \"36d5608a-97e8-4e15-8a53-bb223736daeb\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.150545 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.150524    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4n8w\" (UniqueName: \"kubernetes.io/projected/b6e7f072-03a3-4a4a-a811-f09b2cbae9a6-kube-api-access-t4n8w\") pod \"iptables-alerter-9kprk\" (UID: \"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6\") " pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.151207 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.151183    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbxz7\" (UniqueName: \"kubernetes.io/projected/448d3e76-6d59-46b3-af7d-7714f0fd3772-kube-api-access-nbxz7\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.156898 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.156831    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal" event={"ID":"a09046d52a93c64b9a467e152c6a69f0","Type":"ContainerStarted","Data":"7297647f423382abca4567a70819444e99fb76f539cd50c01a2c2f689a6a32de"}
Jul 06 23:56:18.158139 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.158109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal" event={"ID":"d9d0657b49ada5194a7ba0bead0569b2","Type":"ContainerStarted","Data":"01d223b44caefecb7b9060f899e828387eec0f98e21c74facc71392a3d3877ce"}
Jul 06 23:56:18.231269 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231231    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-system-cni-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231269 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231271    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-cnibin\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231298    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbhpx\" (UniqueName: \"kubernetes.io/projected/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-kube-api-access-jbhpx\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f-konnectivity-ca\") pod \"konnectivity-agent-wqkxt\" (UID: \"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f\") " pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231344    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-socket-dir-parent\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vglh8\" (UniqueName: \"kubernetes.io/projected/b8e53e88-0804-4c75-b30f-263da33307d0-kube-api-access-vglh8\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231374    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-system-cni-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231399    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-cni-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231425    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae566a7-243e-46e2-9bd7-e58e689eb474-tmp-dir\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231449    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-socket-dir-parent\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-multus-certs\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.231530 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231514    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-cni-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.232127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231556    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70b6c117-9122-43d1-aa4f-bee759d72e46-host\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.232127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f-agent-certs\") pod \"konnectivity-agent-wqkxt\" (UID: \"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f\") " pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.232127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231399    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-cnibin\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.232127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bae566a7-243e-46e2-9bd7-e58e689eb474-hosts-file\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.232127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.231688    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-multus-certs\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.232452 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232421    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f-konnectivity-ca\") pod \"konnectivity-agent-wqkxt\" (UID: \"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f\") " pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.232604 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232505    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70b6c117-9122-43d1-aa4f-bee759d72e46-host\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.232666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232626    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.232730 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-system-cni-dir\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.232730 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znk8r\" (UniqueName: \"kubernetes.io/projected/70b6c117-9122-43d1-aa4f-bee759d72e46-kube-api-access-znk8r\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.232730 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-os-release\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.232837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232743    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-hostroot\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.232837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-conf-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.232837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232830    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cnibin\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.232940 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232919    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.233498 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233476    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.233584 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233535    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-system-cni-dir\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.233584 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233574    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-hostroot\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.233684 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233642    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-os-release\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.233720 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233695    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-multus-conf-dir\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.233823 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233805    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bae566a7-243e-46e2-9bd7-e58e689eb474-hosts-file\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.233897 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.232789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cnibin\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.233961 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233942    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.234014 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.233995    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-cni-multus\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234070 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234034    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/70b6c117-9122-43d1-aa4f-bee759d72e46-serviceca\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.234126 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8e53e88-0804-4c75-b30f-263da33307d0-cni-binary-copy\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234126 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-netns\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234238 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234128    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8e53e88-0804-4c75-b30f-263da33307d0-multus-daemon-config\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234238 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-kubelet\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234347 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-os-release\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.234347 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-k8s-cni-cncf-io\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234347 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234283    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bae566a7-243e-46e2-9bd7-e58e689eb474-tmp-dir\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.234347 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-cni-bin\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234532 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234371    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-cni-multus\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234532 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-etc-kubernetes\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234532 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234425    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-etc-kubernetes\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234532 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bkd5\" (UniqueName: \"kubernetes.io/projected/bae566a7-243e-46e2-9bd7-e58e689eb474-kube-api-access-9bkd5\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.234532 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234479    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.234716 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234551    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.234716 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234646    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-k8s-cni-cncf-io\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234801 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234779    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.234873 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234853    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-cni-bin\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234931 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234916    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-run-netns\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.234999 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.234983    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8e53e88-0804-4c75-b30f-263da33307d0-host-var-lib-kubelet\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.235078 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.235063    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-os-release\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.235257 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.235236    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8e53e88-0804-4c75-b30f-263da33307d0-multus-daemon-config\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.235300 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.235243    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8e53e88-0804-4c75-b30f-263da33307d0-cni-binary-copy\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.235575 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.235545    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/70b6c117-9122-43d1-aa4f-bee759d72e46-serviceca\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.236398 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.236372    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f-agent-certs\") pod \"konnectivity-agent-wqkxt\" (UID: \"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f\") " pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.243783 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.243749    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vglh8\" (UniqueName: \"kubernetes.io/projected/b8e53e88-0804-4c75-b30f-263da33307d0-kube-api-access-vglh8\") pod \"multus-9sp9k\" (UID: \"b8e53e88-0804-4c75-b30f-263da33307d0\") " pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.243946 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.243816    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbhpx\" (UniqueName: \"kubernetes.io/projected/f8e3c64b-82b2-4aba-9ac7-8c25345814ba-kube-api-access-jbhpx\") pod \"multus-additional-cni-plugins-96jnk\" (UID: \"f8e3c64b-82b2-4aba-9ac7-8c25345814ba\") " pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.246514 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.246484    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znk8r\" (UniqueName: \"kubernetes.io/projected/70b6c117-9122-43d1-aa4f-bee759d72e46-kube-api-access-znk8r\") pod \"node-ca-w272k\" (UID: \"70b6c117-9122-43d1-aa4f-bee759d72e46\") " pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.246910 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.246887    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bkd5\" (UniqueName: \"kubernetes.io/projected/bae566a7-243e-46e2-9bd7-e58e689eb474-kube-api-access-9bkd5\") pod \"node-resolver-xr9cd\" (UID: \"bae566a7-243e-46e2-9bd7-e58e689eb474\") " pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.317498 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.317455    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:18.326488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.326451    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jbv5t"
Jul 06 23:56:18.335040 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.335002    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm"
Jul 06 23:56:18.343450 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.343407    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9kprk"
Jul 06 23:56:18.350663 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.350630    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xr9cd"
Jul 06 23:56:18.367444 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.367394    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-96jnk"
Jul 06 23:56:18.374647 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.374616    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w272k"
Jul 06 23:56:18.381937 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.381906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:18.388258 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.388226    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9sp9k"
Jul 06 23:56:18.508798 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.508710    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:18.637214 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.637181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:18.637447 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.637333    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:18.637447 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.637334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:18.637447 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.637415    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:19.63739156 +0000 UTC m=+4.002695307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:18.637447 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.637420    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:18.637708 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.637474    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:19.637460741 +0000 UTC m=+4.002764489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:18.738156 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:18.738106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:18.738350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.738274    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:18.738350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.738300    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:18.738350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.738314    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:18.738549 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:18.738382    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:19.738361916 +0000 UTC m=+4.103665643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:19.063477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:19.063429    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:18 +0000 UTC" deadline="2028-04-01 06:22:15.348652446 +0000 UTC"
Jul 06 23:56:19.063477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:19.063466    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15222h25m56.285190265s"
Jul 06 23:56:19.212231 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.212196    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a46aeb8_8183_4cf5_aa3e_c84bcc975b5f.slice/crio-1e69ab16695fb23f6257486ad84f6b1c9cf32d7b5675849b1790cdfb731a8e44 WatchSource:0}: Error finding container 1e69ab16695fb23f6257486ad84f6b1c9cf32d7b5675849b1790cdfb731a8e44: Status 404 returned error can't find the container with id 1e69ab16695fb23f6257486ad84f6b1c9cf32d7b5675849b1790cdfb731a8e44
Jul 06 23:56:19.214376 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.213956    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e7f072_03a3_4a4a_a811_f09b2cbae9a6.slice/crio-e218315b5c8e4729d209edd982d1b4607c18aead0ae094a3d8306a1f1e032143 WatchSource:0}: Error finding container e218315b5c8e4729d209edd982d1b4607c18aead0ae094a3d8306a1f1e032143: Status 404 returned error can't find the container with id e218315b5c8e4729d209edd982d1b4607c18aead0ae094a3d8306a1f1e032143
Jul 06 23:56:19.218978 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.218950    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae566a7_243e_46e2_9bd7_e58e689eb474.slice/crio-416acf63af41fae1efb63099948ab42814b9b62b01a5f8bedd9351b69968d633 WatchSource:0}: Error finding container 416acf63af41fae1efb63099948ab42814b9b62b01a5f8bedd9351b69968d633: Status 404 returned error can't find the container with id 416acf63af41fae1efb63099948ab42814b9b62b01a5f8bedd9351b69968d633
Jul 06 23:56:19.219557 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.219524    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e3c64b_82b2_4aba_9ac7_8c25345814ba.slice/crio-ae817813941d9033f75777ddb1e0a9eb60be4c9b1781d0baf032a98a9186bd04 WatchSource:0}: Error finding container ae817813941d9033f75777ddb1e0a9eb60be4c9b1781d0baf032a98a9186bd04: Status 404 returned error can't find the container with id ae817813941d9033f75777ddb1e0a9eb60be4c9b1781d0baf032a98a9186bd04
Jul 06 23:56:19.220340 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.220320    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d5608a_97e8_4e15_8a53_bb223736daeb.slice/crio-c78680359c46ffbfd0c1715cfdbdcc3cef8329bdca41204ef36a90171eb0cbac WatchSource:0}: Error finding container c78680359c46ffbfd0c1715cfdbdcc3cef8329bdca41204ef36a90171eb0cbac: Status 404 returned error can't find the container with id c78680359c46ffbfd0c1715cfdbdcc3cef8329bdca41204ef36a90171eb0cbac
Jul 06 23:56:19.221411 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.221389    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60418f11_dff3_480e_a1f3_344f9033d62a.slice/crio-0ea3b6bfeabba192455ae8a54bbce9653f718a40a3d1ea79dbcd5462d7bf9865 WatchSource:0}: Error finding container 0ea3b6bfeabba192455ae8a54bbce9653f718a40a3d1ea79dbcd5462d7bf9865: Status 404 returned error can't find the container with id 0ea3b6bfeabba192455ae8a54bbce9653f718a40a3d1ea79dbcd5462d7bf9865
Jul 06 23:56:19.222905 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:19.222813    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b889e54_05f4_424c_8db8_9b28dc7e15c7.slice/crio-43f910c4469ba9adb0c5e8e1c39a445ac900c812b126008798af13ad24a4e023 WatchSource:0}: Error finding container 43f910c4469ba9adb0c5e8e1c39a445ac900c812b126008798af13ad24a4e023: Status 404 returned error can't find the container with id 43f910c4469ba9adb0c5e8e1c39a445ac900c812b126008798af13ad24a4e023
Jul 06 23:56:19.646832 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:19.645697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:19.646832 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:19.646012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:19.646832 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.646201    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:19.646832 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.646272    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:21.64625065 +0000 UTC m=+6.011554381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:19.646832 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.646745    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:19.646832 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.646793    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:21.646781285 +0000 UTC m=+6.012085011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:19.747706 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:19.746906    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:19.747706 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.747139    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:19.747706 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.747165    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:19.747706 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.747179    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:19.747706 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:19.747242    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:21.747223665 +0000 UTC m=+6.112527393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:20.064028 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.063908    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:18 +0000 UTC" deadline="2028-01-23 12:26:17.937348756 +0000 UTC"
Jul 06 23:56:20.064028 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.063948    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13572h29m57.873404925s"
Jul 06 23:56:20.152602 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.151679    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:20.152602 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:20.151818    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:20.152602 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.152312    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:20.152602 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:20.152418    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:20.152602 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.152478    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:20.152602 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:20.152562    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:20.170099 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.170048    2583 generic.go:356] "Generic (PLEG): container finished" podID="d9d0657b49ada5194a7ba0bead0569b2" containerID="7b9dab66f98e96959f8d4c65e13e4d724caa064f16fc837ee14f45626cd49e30" exitCode=0
Jul 06 23:56:20.170285 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.170203    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal" event={"ID":"d9d0657b49ada5194a7ba0bead0569b2","Type":"ContainerDied","Data":"7b9dab66f98e96959f8d4c65e13e4d724caa064f16fc837ee14f45626cd49e30"}
Jul 06 23:56:20.172676 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.172043    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9sp9k" event={"ID":"b8e53e88-0804-4c75-b30f-263da33307d0","Type":"ContainerStarted","Data":"f4e9e946753c1709676280399afe5155d683a72b520d9e8e95dc956f29a830cc"}
Jul 06 23:56:20.175187 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.175107    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w272k" event={"ID":"70b6c117-9122-43d1-aa4f-bee759d72e46","Type":"ContainerStarted","Data":"d8bbdbf0d512c8d29e736677a88454fa5c3d231b7181398470f9e65771d0400b"}
Jul 06 23:56:20.180598 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.180563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm" event={"ID":"60418f11-dff3-480e-a1f3-344f9033d62a","Type":"ContainerStarted","Data":"0ea3b6bfeabba192455ae8a54bbce9653f718a40a3d1ea79dbcd5462d7bf9865"}
Jul 06 23:56:20.190340 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.189878    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"c78680359c46ffbfd0c1715cfdbdcc3cef8329bdca41204ef36a90171eb0cbac"}
Jul 06 23:56:20.191847 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.191702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerStarted","Data":"ae817813941d9033f75777ddb1e0a9eb60be4c9b1781d0baf032a98a9186bd04"}
Jul 06 23:56:20.214196 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.214156    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal" event={"ID":"a09046d52a93c64b9a467e152c6a69f0","Type":"ContainerStarted","Data":"62f8322345a98505650a86375852ab6bc8e52d15867ddd12d1945ebe0420b16a"}
Jul 06 23:56:20.235753 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.235676    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jbv5t" event={"ID":"5b889e54-05f4-424c-8db8-9b28dc7e15c7","Type":"ContainerStarted","Data":"43f910c4469ba9adb0c5e8e1c39a445ac900c812b126008798af13ad24a4e023"}
Jul 06 23:56:20.239661 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.239623    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xr9cd" event={"ID":"bae566a7-243e-46e2-9bd7-e58e689eb474","Type":"ContainerStarted","Data":"416acf63af41fae1efb63099948ab42814b9b62b01a5f8bedd9351b69968d633"}
Jul 06 23:56:20.246370 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.246335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9kprk" event={"ID":"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6","Type":"ContainerStarted","Data":"e218315b5c8e4729d209edd982d1b4607c18aead0ae094a3d8306a1f1e032143"}
Jul 06 23:56:20.248217 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:20.248186    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wqkxt" event={"ID":"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f","Type":"ContainerStarted","Data":"1e69ab16695fb23f6257486ad84f6b1c9cf32d7b5675849b1790cdfb731a8e44"}
Jul 06 23:56:21.282631 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:21.282589    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal" event={"ID":"d9d0657b49ada5194a7ba0bead0569b2","Type":"ContainerStarted","Data":"82d416c43d6ba1a63b7fc19ed945308d305bb958ce5f89bef6112008011f9013"}
Jul 06 23:56:21.306565 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:21.305747    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-68.ec2.internal" podStartSLOduration=4.305725468 podStartE2EDuration="4.305725468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:20.229782316 +0000 UTC m=+4.595086063" watchObservedRunningTime="2026-07-06 23:56:21.305725468 +0000 UTC m=+5.671029215"
Jul 06 23:56:21.306565 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:21.306456    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-68.ec2.internal" podStartSLOduration=4.306441514 podStartE2EDuration="4.306441514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:21.305721808 +0000 UTC m=+5.671025555" watchObservedRunningTime="2026-07-06 23:56:21.306441514 +0000 UTC m=+5.671745259"
Jul 06 23:56:21.664547 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:21.664426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:21.664547 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:21.664480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:21.664772 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.664677    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:21.664823 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.664749    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:25.66472979 +0000 UTC m=+10.030033522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:21.665277 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.665251    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:21.665378 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.665336    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:25.665320196 +0000 UTC m=+10.030623926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:21.765245 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:21.765171    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:21.765417 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.765388    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:21.765417 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.765414    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:21.765571 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.765432    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:21.765571 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:21.765493    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:25.76547377 +0000 UTC m=+10.130777511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:22.155414 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:22.154413    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:22.155414 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:22.154599    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:22.155414 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:22.154959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:22.155414 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:22.155035    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:22.155414 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:22.155184    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:22.155414 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:22.155264    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:24.152733 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:24.152701    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:24.153182 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:24.152799    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:24.153269 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:24.153240    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:24.153401 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:24.153356    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:24.153401 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:24.153388    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:24.153514 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:24.153438    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:25.699770 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:25.699734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:25.700212 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:25.699786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:25.700212 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.699905    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:25.700212 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.699966    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:33.69994712 +0000 UTC m=+18.065250849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:25.700401 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.700382    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:25.700473 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.700440    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:33.700424068 +0000 UTC m=+18.065727815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:25.801007 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:25.800468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:25.801007 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.800678    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:25.801007 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.800701    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:25.801007 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.800715    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:25.801007 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:25.800772    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:33.800754581 +0000 UTC m=+18.166058322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:26.156843 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:26.156765    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:26.156996 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:26.156868    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:26.157176 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:26.157161    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:26.157249 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:26.157233    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:26.157289 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:26.157271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:26.157343 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:26.157330    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:28.151721 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:28.151677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:28.152108 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:28.151809    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:28.152214 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:28.152199    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:28.152286 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:28.152272    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:28.152326 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:28.152317    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:28.152373 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:28.152360    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:30.154662 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:30.154574    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:30.154662 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:30.154609    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:30.155082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:30.154580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:30.155082 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:30.154708    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:30.155082 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:30.154817    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:30.155082 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:30.154907    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:32.151708 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:32.151672    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:32.151708 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:32.151694    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:32.152205 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:32.151672    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:32.152205 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:32.151810    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:32.152205 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:32.152006    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:32.152205 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:32.152067    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:33.758976 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:33.758941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:33.759317 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:33.758982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:33.759317 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.759087    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:33.759317 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.759097    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:33.759317 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.759142    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.759128756 +0000 UTC m=+34.124432480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:33.759317 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.759157    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.759150387 +0000 UTC m=+34.124454111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:33.859824 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:33.859794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:33.860001 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.859952    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:33.860001 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.859969    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:33.860001 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.859977    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:33.860140 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:33.860037    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.86001802 +0000 UTC m=+34.225321763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:34.151949 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:34.151871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:34.152106 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:34.151871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:34.152106 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:34.152003    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:34.152106 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:34.151873    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:34.152106 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:34.152079    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:34.152291 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:34.152165    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:36.152310 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:36.152276    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:36.152786 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:36.152367    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:36.152786 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:36.152766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:36.152898 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:36.152858    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:36.152950 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:36.152908    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:36.153001 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:36.152957    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:38.153168 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.152743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:38.153168 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:38.152865    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:38.155386 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.152910    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:38.155457 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.153059    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:38.155509 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:38.155479    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:38.155608 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:38.155586    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:38.318174 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.318146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9sp9k" event={"ID":"b8e53e88-0804-4c75-b30f-263da33307d0","Type":"ContainerStarted","Data":"360f880aec209f9abf6d3eab71ee21147685ef7794538afc3f0dbba081936fdd"}
Jul 06 23:56:38.319677 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.319645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w272k" event={"ID":"70b6c117-9122-43d1-aa4f-bee759d72e46","Type":"ContainerStarted","Data":"33089f24a27fabbbb4547ac7206ebdabe4e3750b40aaf252fcccd22bd28b3b8c"}
Jul 06 23:56:38.321350 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.321326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm" event={"ID":"60418f11-dff3-480e-a1f3-344f9033d62a","Type":"ContainerStarted","Data":"49ebf00551410d65f8eb66f39cf3777779a06d998127b836f7e4b00419b9774d"}
Jul 06 23:56:38.323336 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.323313    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"f5c21198d1ac262c4339e8bc3263c11c5923742faf5d8ba2160287ddb3aeca25"}
Jul 06 23:56:38.324745 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.324713    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerStarted","Data":"c50e1cbb6b5faf6c7a1ca59390d272028cb0162d40323e5870dfbca5be1e8a12"}
Jul 06 23:56:38.326105 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.326086    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jbv5t" event={"ID":"5b889e54-05f4-424c-8db8-9b28dc7e15c7","Type":"ContainerStarted","Data":"c6fc854427ffed474b0ab4a25d77d4b6fcf0787084f896c95cdf8c49f136e697"}
Jul 06 23:56:38.327409 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.327388    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xr9cd" event={"ID":"bae566a7-243e-46e2-9bd7-e58e689eb474","Type":"ContainerStarted","Data":"95c99586fcfe90d85101124bf3f10e549277eb39682237dd1099474dba475676"}
Jul 06 23:56:38.328586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.328569    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wqkxt" event={"ID":"8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f","Type":"ContainerStarted","Data":"83f35a60a44d56408f5c7616500f1ac1493d56e94e28a94aa2357dfd2d4d3314"}
Jul 06 23:56:38.351291 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.351253    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9sp9k" podStartSLOduration=3.514149071 podStartE2EDuration="22.351241601s" totalImagesPullingTime="18.83709253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:38.335711199 +0000 UTC m=+22.701014945" watchObservedRunningTime="2026-07-06 23:56:38.351241601 +0000 UTC m=+22.716545347"
Jul 06 23:56:38.351625 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.351603    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xr9cd" podStartSLOduration=3.533314308 podStartE2EDuration="22.351594932s" totalImagesPullingTime="18.818280624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:38.350918827 +0000 UTC m=+22.716222573" watchObservedRunningTime="2026-07-06 23:56:38.351594932 +0000 UTC m=+22.716898678"
Jul 06 23:56:38.391000 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.390959    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w272k" podStartSLOduration=3.602874343 podStartE2EDuration="22.390946195s" totalImagesPullingTime="18.788071852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:38.390662367 +0000 UTC m=+22.755966113" watchObservedRunningTime="2026-07-06 23:56:38.390946195 +0000 UTC m=+22.756249966"
Jul 06 23:56:38.409082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:38.409014    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wqkxt" podStartSLOduration=3.9706787930000003 podStartE2EDuration="22.408993312s" totalImagesPullingTime="18.438314519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:38.408551435 +0000 UTC m=+22.773855181" watchObservedRunningTime="2026-07-06 23:56:38.408993312 +0000 UTC m=+22.774297058"
Jul 06 23:56:39.332456 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.332268    2583 generic.go:356] "Generic (PLEG): container finished" podID="f8e3c64b-82b2-4aba-9ac7-8c25345814ba" containerID="c50e1cbb6b5faf6c7a1ca59390d272028cb0162d40323e5870dfbca5be1e8a12" exitCode=0
Jul 06 23:56:39.333224 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.332353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerDied","Data":"c50e1cbb6b5faf6c7a1ca59390d272028cb0162d40323e5870dfbca5be1e8a12"}
Jul 06 23:56:39.335646 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.335630    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 06 23:56:39.336022 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.336001    2583 generic.go:356] "Generic (PLEG): container finished" podID="36d5608a-97e8-4e15-8a53-bb223736daeb" containerID="fc23fcef95e803c8d042956361211263ebfc5dcdfd1ded7fd7a39baeb1dabed1" exitCode=1
Jul 06 23:56:39.336104 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.336083    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"1f0e70a3f4f5a2a5ebd4e87ac7e2ba78dae6c66a51c048be9dd758bd0270ab40"}
Jul 06 23:56:39.336179 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.336155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"de5879fa3ce75e2420d795453e202b3da60f98873aff81a7eb41211a0256a905"}
Jul 06 23:56:39.336228 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.336204    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"bf2a87b14d674687f8fd444e1497a957507791ee086b8e4d9ac6c9fe2a29e222"}
Jul 06 23:56:39.336284 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.336232    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"11285c5a2792160d94e37fd3768898015b1a9065dccf7d3c2adff9ab61175098"}
Jul 06 23:56:39.336284 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.336270    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerDied","Data":"fc23fcef95e803c8d042956361211263ebfc5dcdfd1ded7fd7a39baeb1dabed1"}
Jul 06 23:56:39.358375 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.358341    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jbv5t" podStartSLOduration=4.551451192 podStartE2EDuration="23.358328469s" totalImagesPullingTime="18.806877277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:38.433273882 +0000 UTC m=+22.798577631" watchObservedRunningTime="2026-07-06 23:56:39.358328469 +0000 UTC m=+23.723632214"
Jul 06 23:56:39.438578 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:39.438557    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 23:56:40.082169 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.082048    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:56:39.438572995Z","UUID":"31935bb7-de09-4d64-af2a-7bb74c0deb74","Handler":null,"Name":"","Endpoint":""}
Jul 06 23:56:40.084888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.084861    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 23:56:40.085019 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.084910    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 23:56:40.151663 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.151634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:40.151663 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.151658    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:40.151893 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.151742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:40.151893 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:40.151762    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:40.151893 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:40.151828    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:40.151999 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:40.151914    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:40.339863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.339776    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9kprk" event={"ID":"b6e7f072-03a3-4a4a-a811-f09b2cbae9a6","Type":"ContainerStarted","Data":"9aae41907d583d1cc89c987c7fb46f041ad7fd6eebc264b623397a36592c2c78"}
Jul 06 23:56:40.341643 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:40.341611    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm" event={"ID":"60418f11-dff3-480e-a1f3-344f9033d62a","Type":"ContainerStarted","Data":"4e6bc98b876d41b4ce4a5fb2a51edeb0334620822c7f84110236dc6c4ec4b49a"}
Jul 06 23:56:41.349198 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:41.349108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm" event={"ID":"60418f11-dff3-480e-a1f3-344f9033d62a","Type":"ContainerStarted","Data":"3f5b6a672da75f4e4e3b821779d30548d7b4242b79df68ee4cfe67a768dfb30c"}
Jul 06 23:56:41.352403 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:41.352383    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 06 23:56:41.352828 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:41.352803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"a1d644a878d58a8af0f6f593e263d3fa112861375a6a10205cecf05aa452c74d"}
Jul 06 23:56:41.370343 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:41.370247    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9kprk" podStartSLOduration=6.5541808580000005 podStartE2EDuration="25.370234276s" totalImagesPullingTime="18.816053418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:40.354551619 +0000 UTC m=+24.719855365" watchObservedRunningTime="2026-07-06 23:56:41.370234276 +0000 UTC m=+25.735538069"
Jul 06 23:56:42.151952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.151922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:42.151952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.151946    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:42.152219 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:42.152062    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:42.152219 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.152112    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:42.152328 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:42.152209    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:42.152328 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:42.152304    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:42.217278 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.217249    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:42.217971 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.217940    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:42.233419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.233375    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4mjcm" podStartSLOduration=5.441430654 podStartE2EDuration="26.233361691s" totalImagesPullingTime="20.791931037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:41.370208465 +0000 UTC m=+25.735512212" watchObservedRunningTime="2026-07-06 23:56:42.233361691 +0000 UTC m=+26.598665436"
Jul 06 23:56:42.355164 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.355138    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:42.355941 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:42.355921    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wqkxt"
Jul 06 23:56:43.363921 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.363307    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 06 23:56:43.363921 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.363843    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"230c5f614e27cf4b0df4dddabf7fa6db9e44d3fe30f6df9c4609f2ba67af3caa"}
Jul 06 23:56:43.365311 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.364122    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:43.365311 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.364192    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:43.365311 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.364238    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:43.365311 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.364282    2583 scope.go:117] "RemoveContainer" containerID="fc23fcef95e803c8d042956361211263ebfc5dcdfd1ded7fd7a39baeb1dabed1"
Jul 06 23:56:43.384880 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.384792    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:43.386441 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:43.386405    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:56:44.152231 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.152033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:44.152376 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.152033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:44.152376 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:44.152310    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:44.152476 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:44.152431    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:44.152476 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.152110    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:44.152614 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:44.152593    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:44.369105 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.369079    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 06 23:56:44.369465 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.369445    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" event={"ID":"36d5608a-97e8-4e15-8a53-bb223736daeb","Type":"ContainerStarted","Data":"dccf77dc2f4ca153c933fcf41003be0efcde513c585d8bb55ae146e0dffdf016"}
Jul 06 23:56:44.371285 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.371258    2583 generic.go:356] "Generic (PLEG): container finished" podID="f8e3c64b-82b2-4aba-9ac7-8c25345814ba" containerID="1a4056de34d4f385b6797f2979dec049f7a115b943eda9500a806d1b950f5772" exitCode=0
Jul 06 23:56:44.371385 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.371339    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerDied","Data":"1a4056de34d4f385b6797f2979dec049f7a115b943eda9500a806d1b950f5772"}
Jul 06 23:56:44.399412 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:44.399370    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs" podStartSLOduration=9.56526307 podStartE2EDuration="28.399356117s" totalImagesPullingTime="18.834093047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:44.397657721 +0000 UTC m=+28.762961467" watchObservedRunningTime="2026-07-06 23:56:44.399356117 +0000 UTC m=+28.764659864"
Jul 06 23:56:45.251653 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.251439    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6wfrf"]
Jul 06 23:56:45.251758 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.251672    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:45.251758 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:45.251746    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:45.254388 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.254365    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zkcfs"]
Jul 06 23:56:45.254553 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.254518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:45.254666 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:45.254648    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:45.255095 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.255075    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fwwp7"]
Jul 06 23:56:45.255169 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.255159    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:45.255266 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:45.255249    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:45.374974 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.374943    2583 generic.go:356] "Generic (PLEG): container finished" podID="f8e3c64b-82b2-4aba-9ac7-8c25345814ba" containerID="1e5497b50ed3f371c0596219beeda78785c5b5d47222c5397d66e565ef0123cc" exitCode=0
Jul 06 23:56:45.375358 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:45.375035    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerDied","Data":"1e5497b50ed3f371c0596219beeda78785c5b5d47222c5397d66e565ef0123cc"}
Jul 06 23:56:46.379432 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:46.379354    2583 generic.go:356] "Generic (PLEG): container finished" podID="f8e3c64b-82b2-4aba-9ac7-8c25345814ba" containerID="812baaa29e2e5ca92dc31250836de81dd434bde508c0ced76edd9fd72cf0d894" exitCode=0
Jul 06 23:56:46.379432 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:46.379394    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerDied","Data":"812baaa29e2e5ca92dc31250836de81dd434bde508c0ced76edd9fd72cf0d894"}
Jul 06 23:56:47.151435 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:47.151400    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:47.151435 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:47.151427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:47.151621 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:47.151454    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:47.151621 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:47.151600    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:47.151769 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:47.151741    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:47.151851 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:47.151828    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:49.151178 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:49.151150    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:49.151909 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:49.151150    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:49.151909 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.151257    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwwp7" podUID="448d3e76-6d59-46b3-af7d-7714f0fd3772"
Jul 06 23:56:49.151909 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:49.151150    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:49.151909 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.151342    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wfrf" podUID="37d6930f-b6fa-4f56-b860-ab8fdb517d76"
Jul 06 23:56:49.151909 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.151448    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkcfs" podUID="da53e071-0353-4538-8837-240274613bc2"
Jul 06 23:56:49.786134 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:49.786041    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:49.786134 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:49.786111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:49.786334 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.786171    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:49.786334 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.786211    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:49.786334 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.786240    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs podName:448d3e76-6d59-46b3-af7d-7714f0fd3772 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:21.786223355 +0000 UTC m=+66.151527081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs") pod "network-metrics-daemon-fwwp7" (UID: "448d3e76-6d59-46b3-af7d-7714f0fd3772") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:49.786334 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.786257    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret podName:da53e071-0353-4538-8837-240274613bc2 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:21.786246372 +0000 UTC m=+66.151550096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret") pod "global-pull-secret-syncer-zkcfs" (UID: "da53e071-0353-4538-8837-240274613bc2") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:49.886929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:49.886895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:49.887118 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.887063    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:49.887118 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.887081    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:49.887118 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.887091    2583 projected.go:196] Error preparing data for projected volume kube-api-access-7hrc5 for pod openshift-network-diagnostics/network-check-target-6wfrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:49.887276 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:49.887146    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5 podName:37d6930f-b6fa-4f56-b860-ab8fdb517d76 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:21.887133009 +0000 UTC m=+66.252436734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7hrc5" (UniqueName: "kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5") pod "network-check-target-6wfrf" (UID: "37d6930f-b6fa-4f56-b860-ab8fdb517d76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:50.545345 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:50.545316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xr9cd_bae566a7-243e-46e2-9bd7-e58e689eb474/dns-node-resolver/0.log"
Jul 06 23:56:50.971060 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:50.971035    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-68.ec2.internal" event="NodeReady"
Jul 06 23:56:50.971222 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:50.971160    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 23:56:51.015452 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.015426    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4p5cx"]
Jul 06 23:56:51.028787 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.028764    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bvdb7"]
Jul 06 23:56:51.028900 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.028881    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.032668 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.032152    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dkpzw\""
Jul 06 23:56:51.032668 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.032286    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 06 23:56:51.032668 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.032434    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 06 23:56:51.032668 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.032569    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 06 23:56:51.032668 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.032590    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 06 23:56:51.042823 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.042802    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mjk9s"]
Jul 06 23:56:51.042917 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.042904    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.045468 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.045448    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c6s9h\""
Jul 06 23:56:51.045672 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.045648    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 23:56:51.045777 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.045630    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 23:56:51.063127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.063097    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4p5cx"]
Jul 06 23:56:51.063298 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.063135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvdb7"]
Jul 06 23:56:51.063298 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.063163    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mjk9s"]
Jul 06 23:56:51.063298 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.063195    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q4mbb"]
Jul 06 23:56:51.063298 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.063210    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.065594 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.065577    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 23:56:51.066221 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.066205    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 23:56:51.066443 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.066410    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-phqcp\""
Jul 06 23:56:51.066689 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.066675    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 23:56:51.066781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.066688    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 23:56:51.085277 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.085230    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q4mbb"]
Jul 06 23:56:51.085361 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.085306    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.087698 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.087682    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 23:56:51.087796 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.087699    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5xctx\""
Jul 06 23:56:51.087796 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.087714    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 23:56:51.087796 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.087731    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 23:56:51.151668 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.151642    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:56:51.151808 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.151735    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:56:51.151808 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.151760    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:56:51.154142 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.154118    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:56:51.154253 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.154234    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:56:51.154315 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.154272    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:56:51.154406 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.154377    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:56:51.154406 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.154395    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6v6rl\""
Jul 06 23:56:51.154550 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.154428    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m54k2\""
Jul 06 23:56:51.196658 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196638    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-data-volume\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.196751 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.196751 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9046d0c8-a0e1-4641-bbbd-b7f10d639dcb-cert\") pod \"ingress-canary-q4mbb\" (UID: \"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb\") " pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.196751 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhfw\" (UniqueName: \"kubernetes.io/projected/9046d0c8-a0e1-4641-bbbd-b7f10d639dcb-kube-api-access-bdhfw\") pod \"ingress-canary-q4mbb\" (UID: \"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb\") " pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.196751 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.196751 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdqh9\" (UniqueName: \"kubernetes.io/projected/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-api-access-cdqh9\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.196912 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196781    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-crio-socket\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.196912 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196864    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zw5\" (UniqueName: \"kubernetes.io/projected/adb4d8be-376d-41c1-93a1-36ccaa823c3e-kube-api-access-m8zw5\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.196912 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196894    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.197035 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/adb4d8be-376d-41c1-93a1-36ccaa823c3e-signing-cabundle\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.197035 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196938    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6896410a-be62-4c56-a0ef-13dca1e18d5c-config-volume\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.197035 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.196957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6896410a-be62-4c56-a0ef-13dca1e18d5c-tmp-dir\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.197035 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.197010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/adb4d8be-376d-41c1-93a1-36ccaa823c3e-signing-key\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.197190 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.197044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg4c6\" (UniqueName: \"kubernetes.io/projected/6896410a-be62-4c56-a0ef-13dca1e18d5c-kube-api-access-cg4c6\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.297999 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.297966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-crio-socket\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298032    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zw5\" (UniqueName: \"kubernetes.io/projected/adb4d8be-376d-41c1-93a1-36ccaa823c3e-kube-api-access-m8zw5\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.298177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.298177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/adb4d8be-376d-41c1-93a1-36ccaa823c3e-signing-cabundle\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.298177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298132    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6896410a-be62-4c56-a0ef-13dca1e18d5c-config-volume\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.298177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6896410a-be62-4c56-a0ef-13dca1e18d5c-tmp-dir\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.298177 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/adb4d8be-376d-41c1-93a1-36ccaa823c3e-signing-key\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.298453 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298196    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-crio-socket\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298453 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.298211    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:51.298453 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg4c6\" (UniqueName: \"kubernetes.io/projected/6896410a-be62-4c56-a0ef-13dca1e18d5c-kube-api-access-cg4c6\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.298453 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.298276    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls podName:6896410a-be62-4c56-a0ef-13dca1e18d5c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.798257111 +0000 UTC m=+36.163560849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls") pod "dns-default-bvdb7" (UID: "6896410a-be62-4c56-a0ef-13dca1e18d5c") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:51.298453 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298344    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-data-volume\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298453 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298372    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298765 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9046d0c8-a0e1-4641-bbbd-b7f10d639dcb-cert\") pod \"ingress-canary-q4mbb\" (UID: \"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb\") " pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.298765 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298656    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhfw\" (UniqueName: \"kubernetes.io/projected/9046d0c8-a0e1-4641-bbbd-b7f10d639dcb-kube-api-access-bdhfw\") pod \"ingress-canary-q4mbb\" (UID: \"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb\") " pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.298765 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298765 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdqh9\" (UniqueName: \"kubernetes.io/projected/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-api-access-cdqh9\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298945 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298791    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-data-volume\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.298945 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.298621    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:51.298945 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298850    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6896410a-be62-4c56-a0ef-13dca1e18d5c-config-volume\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.298945 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.298876    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.798859703 +0000 UTC m=+36.164163441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:51.298945 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.298890    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:51.298945 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.298899    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/adb4d8be-376d-41c1-93a1-36ccaa823c3e-signing-cabundle\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.299146 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.298953    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.798939557 +0000 UTC m=+36.164243282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:51.299146 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.299098    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6896410a-be62-4c56-a0ef-13dca1e18d5c-tmp-dir\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.302420 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.302395    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/adb4d8be-376d-41c1-93a1-36ccaa823c3e-signing-key\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.302618 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.302601    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9046d0c8-a0e1-4641-bbbd-b7f10d639dcb-cert\") pod \"ingress-canary-q4mbb\" (UID: \"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb\") " pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.312208 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.312189    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhfw\" (UniqueName: \"kubernetes.io/projected/9046d0c8-a0e1-4641-bbbd-b7f10d639dcb-kube-api-access-bdhfw\") pod \"ingress-canary-q4mbb\" (UID: \"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb\") " pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.313786 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.313751    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg4c6\" (UniqueName: \"kubernetes.io/projected/6896410a-be62-4c56-a0ef-13dca1e18d5c-kube-api-access-cg4c6\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.314386 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.314362    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdqh9\" (UniqueName: \"kubernetes.io/projected/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-api-access-cdqh9\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.315325 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.315306    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zw5\" (UniqueName: \"kubernetes.io/projected/adb4d8be-376d-41c1-93a1-36ccaa823c3e-kube-api-access-m8zw5\") pod \"service-ca-758f599c5-4p5cx\" (UID: \"adb4d8be-376d-41c1-93a1-36ccaa823c3e\") " pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.325962 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.325945    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w272k_70b6c117-9122-43d1-aa4f-bee759d72e46/node-ca/0.log"
Jul 06 23:56:51.342156 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.342087    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4p5cx"
Jul 06 23:56:51.392790 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.392754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q4mbb"
Jul 06 23:56:51.398462 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.398438    2583 generic.go:356] "Generic (PLEG): container finished" podID="f8e3c64b-82b2-4aba-9ac7-8c25345814ba" containerID="307e0e16f0ee71353df48256e14c54f8df51291f147e77a8d9e7853e636f7448" exitCode=0
Jul 06 23:56:51.398599 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.398484    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerDied","Data":"307e0e16f0ee71353df48256e14c54f8df51291f147e77a8d9e7853e636f7448"}
Jul 06 23:56:51.534108 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.534071    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4p5cx"]
Jul 06 23:56:51.537319 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:51.537296    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadb4d8be_376d_41c1_93a1_36ccaa823c3e.slice/crio-38a0626fe4f7e05cd7e9ffc19e784e873e95ff3c00d3f53c2747ece558411327 WatchSource:0}: Error finding container 38a0626fe4f7e05cd7e9ffc19e784e873e95ff3c00d3f53c2747ece558411327: Status 404 returned error can't find the container with id 38a0626fe4f7e05cd7e9ffc19e784e873e95ff3c00d3f53c2747ece558411327
Jul 06 23:56:51.552769 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.552737    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q4mbb"]
Jul 06 23:56:51.556812 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:51.556789    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9046d0c8_a0e1_4641_bbbd_b7f10d639dcb.slice/crio-813ea20f49b4e5b2a8d2b666a13570772c76df97d8897060c38d5fa2aa56d18e WatchSource:0}: Error finding container 813ea20f49b4e5b2a8d2b666a13570772c76df97d8897060c38d5fa2aa56d18e: Status 404 returned error can't find the container with id 813ea20f49b4e5b2a8d2b666a13570772c76df97d8897060c38d5fa2aa56d18e
Jul 06 23:56:51.802752 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.802671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.802752 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.802713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:51.802770    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.802824    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.802857    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.802878    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.802889    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:52.802873753 +0000 UTC m=+37.168177476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.802908    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:52.802895556 +0000 UTC m=+37.168199283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:51.802971 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:51.802922    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls podName:6896410a-be62-4c56-a0ef-13dca1e18d5c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:52.802915934 +0000 UTC m=+37.168219658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls") pod "dns-default-bvdb7" (UID: "6896410a-be62-4c56-a0ef-13dca1e18d5c") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:52.405874 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.405839    2583 generic.go:356] "Generic (PLEG): container finished" podID="f8e3c64b-82b2-4aba-9ac7-8c25345814ba" containerID="ce1533375633f0a0e547222562c74658267c364dbe5da08e8f6ae8c45afbee4c" exitCode=0
Jul 06 23:56:52.406189 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.405916    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerDied","Data":"ce1533375633f0a0e547222562c74658267c364dbe5da08e8f6ae8c45afbee4c"}
Jul 06 23:56:52.408273 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.408249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q4mbb" event={"ID":"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb","Type":"ContainerStarted","Data":"813ea20f49b4e5b2a8d2b666a13570772c76df97d8897060c38d5fa2aa56d18e"}
Jul 06 23:56:52.410103 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.410059    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4p5cx" event={"ID":"adb4d8be-376d-41c1-93a1-36ccaa823c3e","Type":"ContainerStarted","Data":"38a0626fe4f7e05cd7e9ffc19e784e873e95ff3c00d3f53c2747ece558411327"}
Jul 06 23:56:52.810957 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.810720    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:52.810957 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.810924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:52.810976    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:52.811079    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:52.811132    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls podName:6896410a-be62-4c56-a0ef-13dca1e18d5c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:54.811114456 +0000 UTC m=+39.176418181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls") pod "dns-default-bvdb7" (UID: "6896410a-be62-4c56-a0ef-13dca1e18d5c") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:52.811181    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:52.811203    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:54.811196423 +0000 UTC m=+39.176500147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:52.811236    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:52.811410 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:52.811255    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:54.811249414 +0000 UTC m=+39.176553138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:53.416846 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:53.416811    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-96jnk" event={"ID":"f8e3c64b-82b2-4aba-9ac7-8c25345814ba","Type":"ContainerStarted","Data":"c4bfd76ae2c78b4d946b9393a2f5517675fae863410cb0570bd26ee8d4328ac4"}
Jul 06 23:56:53.443197 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:53.443129    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-96jnk" podStartSLOduration=9.252558663 podStartE2EDuration="37.443110777s" totalImagesPullingTime="28.190552114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:53.440867311 +0000 UTC m=+37.806171070" watchObservedRunningTime="2026-07-06 23:56:53.443110777 +0000 UTC m=+37.808414520"
Jul 06 23:56:54.827507 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:54.827468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:54.827507 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:54.827509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:54.828350 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:54.827571    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:54.828350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:54.827649    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:54.828350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:54.827665    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:54.828350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:54.827725    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:56:58.827704986 +0000 UTC m=+43.193008713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:54.828350 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:54.827742    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls podName:6896410a-be62-4c56-a0ef-13dca1e18d5c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:58.827734221 +0000 UTC m=+43.193037945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls") pod "dns-default-bvdb7" (UID: "6896410a-be62-4c56-a0ef-13dca1e18d5c") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:54.828350 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:54.828136    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:55.423703 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:55.423668    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q4mbb" event={"ID":"9046d0c8-a0e1-4641-bbbd-b7f10d639dcb","Type":"ContainerStarted","Data":"8eee439a00cec41eccedfb07ffa96298436b48d5c4643966e9a9dd9136b00772"}
Jul 06 23:56:55.425019 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:55.424998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4p5cx" event={"ID":"adb4d8be-376d-41c1-93a1-36ccaa823c3e","Type":"ContainerStarted","Data":"a3bb239442ff5eb8b746243695c24d4d4a3d41eb663cc8b3845b7c3dfcdbba19"}
Jul 06 23:56:55.455440 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:55.455392    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q4mbb" podStartSLOduration=1.462642628 podStartE2EDuration="4.45537898s" totalImagesPullingTime="2.992736352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.451861251 +0000 UTC m=+39.817164998" watchObservedRunningTime="2026-07-06 23:56:55.45537898 +0000 UTC m=+39.820682725"
Jul 06 23:56:55.504084 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:55.504023    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-4p5cx" podStartSLOduration=2.497730823 podStartE2EDuration="5.503985662s" totalImagesPullingTime="3.006254839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.503393195 +0000 UTC m=+39.868697003" watchObservedRunningTime="2026-07-06 23:56:55.503985662 +0000 UTC m=+39.869289409"
Jul 06 23:56:58.857382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:58.857342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:56:58.857826 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:58.857404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:58.857826 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:58.857488    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:58.857826 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:56:58.857580    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls podName:93cdaebe-f266-44cb-b2f4-c2d3c9081cdb nodeName:}" failed. No retries permitted until 2026-07-06 23:57:06.857562869 +0000 UTC m=+51.222866606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjk9s" (UID: "93cdaebe-f266-44cb-b2f4-c2d3c9081cdb") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:58.861401 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:58.861374    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6896410a-be62-4c56-a0ef-13dca1e18d5c-metrics-tls\") pod \"dns-default-bvdb7\" (UID: \"6896410a-be62-4c56-a0ef-13dca1e18d5c\") " pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:59.151586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:59.151467    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:56:59.327709 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:59.327674    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvdb7"]
Jul 06 23:56:59.331010 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:56:59.330979    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6896410a_be62_4c56_a0ef_13dca1e18d5c.slice/crio-502da82bf3c1a1d431067a697e86794fe87bdc6790c139754ad4ea1258bcd6db WatchSource:0}: Error finding container 502da82bf3c1a1d431067a697e86794fe87bdc6790c139754ad4ea1258bcd6db: Status 404 returned error can't find the container with id 502da82bf3c1a1d431067a697e86794fe87bdc6790c139754ad4ea1258bcd6db
Jul 06 23:56:59.434898 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:56:59.434822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvdb7" event={"ID":"6896410a-be62-4c56-a0ef-13dca1e18d5c","Type":"ContainerStarted","Data":"502da82bf3c1a1d431067a697e86794fe87bdc6790c139754ad4ea1258bcd6db"}
Jul 06 23:57:01.442121 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:01.442062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvdb7" event={"ID":"6896410a-be62-4c56-a0ef-13dca1e18d5c","Type":"ContainerStarted","Data":"dac9e1df526a5e15b8b2500f104f067f2ccf4d588559febde476c80b4d4d3748"}
Jul 06 23:57:02.447160 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:02.447127    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvdb7" event={"ID":"6896410a-be62-4c56-a0ef-13dca1e18d5c","Type":"ContainerStarted","Data":"bfe4d4a92b5428f0bc93d58eccd2be361ff670adb0d6ea5e2d498d31c40b964c"}
Jul 06 23:57:02.447595 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:02.447279    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:57:02.474056 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:02.474014    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bvdb7" podStartSLOduration=9.53936035 podStartE2EDuration="11.473999551s" totalImagesPullingTime="1.934639201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:02.472745796 +0000 UTC m=+46.838049546" watchObservedRunningTime="2026-07-06 23:57:02.473999551 +0000 UTC m=+46.839303299"
Jul 06 23:57:06.915043 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:06.915004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:57:06.917771 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:06.917748    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93cdaebe-f266-44cb-b2f4-c2d3c9081cdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjk9s\" (UID: \"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb\") " pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:57:06.971051 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:06.971023    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mjk9s"
Jul 06 23:57:07.115807 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:07.115780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mjk9s"]
Jul 06 23:57:07.119407 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:07.119384    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cdaebe_f266_44cb_b2f4_c2d3c9081cdb.slice/crio-a61943f62b6f7483bd9b06fa3e940ebe25c3b7b89765c10c0c36fa36fdd93227 WatchSource:0}: Error finding container a61943f62b6f7483bd9b06fa3e940ebe25c3b7b89765c10c0c36fa36fdd93227: Status 404 returned error can't find the container with id a61943f62b6f7483bd9b06fa3e940ebe25c3b7b89765c10c0c36fa36fdd93227
Jul 06 23:57:07.459811 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:07.459782    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjk9s" event={"ID":"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb","Type":"ContainerStarted","Data":"a61943f62b6f7483bd9b06fa3e940ebe25c3b7b89765c10c0c36fa36fdd93227"}
Jul 06 23:57:11.473364 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:11.473317    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjk9s" event={"ID":"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb","Type":"ContainerStarted","Data":"d00d191642638860f3e9efa54249f31146581a4ba2da17fdbd24a36c9b84a441"}
Jul 06 23:57:12.452699 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:12.452668    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bvdb7"
Jul 06 23:57:12.479020 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:12.478985    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjk9s" event={"ID":"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb","Type":"ContainerStarted","Data":"e7394784fd2f2be5482099c7c94a63206dabb77eb2230b88064aae4c36eb7bb2"}
Jul 06 23:57:13.473927 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.473893    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"]
Jul 06 23:57:13.479833 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.479809    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.487165 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.486447    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ftjpj\""
Jul 06 23:57:13.487165 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.486742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 23:57:13.487165 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.486755    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 23:57:13.487165 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.487035    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 23:57:13.487564 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.487520    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 23:57:13.499883 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.499854    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"]
Jul 06 23:57:13.561265 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.561234    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dqwb\" (UniqueName: \"kubernetes.io/projected/0e581d6c-c82e-442c-aee9-531aae945405-kube-api-access-7dqwb\") pod \"managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6\" (UID: \"0e581d6c-c82e-442c-aee9-531aae945405\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.561434 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.561355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e581d6c-c82e-442c-aee9-531aae945405-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6\" (UID: \"0e581d6c-c82e-442c-aee9-531aae945405\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.590311 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.590272    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64f66b6665-lpmjm"]
Jul 06 23:57:13.594316 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.594294    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.610982 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.610958    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qgl2r\""
Jul 06 23:57:13.611112 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.610994    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 23:57:13.612036 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.612016    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 23:57:13.628471 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.628445    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f66b6665-lpmjm"]
Jul 06 23:57:13.631842 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.631821    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 23:57:13.640027 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.639996    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 23:57:13.662020 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.661961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqwb\" (UniqueName: \"kubernetes.io/projected/0e581d6c-c82e-442c-aee9-531aae945405-kube-api-access-7dqwb\") pod \"managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6\" (UID: \"0e581d6c-c82e-442c-aee9-531aae945405\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.662141 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662048    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b7ac5bca-caa4-4fbc-bf90-de9897830455-ca-trust-extracted\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662141 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662073    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b7ac5bca-caa4-4fbc-bf90-de9897830455-registry-certificates\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662141 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662098    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b7ac5bca-caa4-4fbc-bf90-de9897830455-installation-pull-secrets\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662305 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7ac5bca-caa4-4fbc-bf90-de9897830455-trusted-ca\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662305 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662224    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsxgp\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-kube-api-access-zsxgp\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662305 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662252    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b7ac5bca-caa4-4fbc-bf90-de9897830455-image-registry-private-configuration\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662305 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-registry-tls\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662486 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662314    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-bound-sa-token\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.662486 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.662335    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e581d6c-c82e-442c-aee9-531aae945405-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6\" (UID: \"0e581d6c-c82e-442c-aee9-531aae945405\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.665254 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.665233    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e581d6c-c82e-442c-aee9-531aae945405-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6\" (UID: \"0e581d6c-c82e-442c-aee9-531aae945405\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.680276 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.680251    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dqwb\" (UniqueName: \"kubernetes.io/projected/0e581d6c-c82e-442c-aee9-531aae945405-kube-api-access-7dqwb\") pod \"managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6\" (UID: \"0e581d6c-c82e-442c-aee9-531aae945405\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.763102 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b7ac5bca-caa4-4fbc-bf90-de9897830455-ca-trust-extracted\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763107    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b7ac5bca-caa4-4fbc-bf90-de9897830455-registry-certificates\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b7ac5bca-caa4-4fbc-bf90-de9897830455-installation-pull-secrets\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7ac5bca-caa4-4fbc-bf90-de9897830455-trusted-ca\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxgp\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-kube-api-access-zsxgp\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b7ac5bca-caa4-4fbc-bf90-de9897830455-image-registry-private-configuration\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-registry-tls\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763602 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763285    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-bound-sa-token\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.763602 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.763555    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b7ac5bca-caa4-4fbc-bf90-de9897830455-ca-trust-extracted\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.764286 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.764229    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b7ac5bca-caa4-4fbc-bf90-de9897830455-registry-certificates\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.764437 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.764407    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7ac5bca-caa4-4fbc-bf90-de9897830455-trusted-ca\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.766091 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.766067    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b7ac5bca-caa4-4fbc-bf90-de9897830455-image-registry-private-configuration\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.766387 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.766369    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b7ac5bca-caa4-4fbc-bf90-de9897830455-installation-pull-secrets\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.766571 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.766551    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-registry-tls\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.775721 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.775697    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxgp\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-kube-api-access-zsxgp\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.781000 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.780979    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7ac5bca-caa4-4fbc-bf90-de9897830455-bound-sa-token\") pod \"image-registry-64f66b6665-lpmjm\" (UID: \"b7ac5bca-caa4-4fbc-bf90-de9897830455\") " pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:13.799913 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.799892    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"
Jul 06 23:57:13.905650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:13.905616    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:14.123107 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.123018    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f66b6665-lpmjm"]
Jul 06 23:57:14.125440 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:14.125413    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ac5bca_caa4_4fbc_bf90_de9897830455.slice/crio-5fb81f2abecd735717b6840cfdd37d89041312079a63b64224aaf1810fcdc0ae WatchSource:0}: Error finding container 5fb81f2abecd735717b6840cfdd37d89041312079a63b64224aaf1810fcdc0ae: Status 404 returned error can't find the container with id 5fb81f2abecd735717b6840cfdd37d89041312079a63b64224aaf1810fcdc0ae
Jul 06 23:57:14.131956 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.131936    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6"]
Jul 06 23:57:14.135047 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:14.135022    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e581d6c_c82e_442c_aee9_531aae945405.slice/crio-2612c987103aefc7092c0eec8e744cff450528ddec9769e5f3b557ca42bca47a WatchSource:0}: Error finding container 2612c987103aefc7092c0eec8e744cff450528ddec9769e5f3b557ca42bca47a: Status 404 returned error can't find the container with id 2612c987103aefc7092c0eec8e744cff450528ddec9769e5f3b557ca42bca47a
Jul 06 23:57:14.489172 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.489131    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjk9s" event={"ID":"93cdaebe-f266-44cb-b2f4-c2d3c9081cdb","Type":"ContainerStarted","Data":"9ec115e811eb12f499e6f6fdc767456677bbbb9b938e92b3e60745c194b4fd49"}
Jul 06 23:57:14.490247 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.490218    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6" event={"ID":"0e581d6c-c82e-442c-aee9-531aae945405","Type":"ContainerStarted","Data":"2612c987103aefc7092c0eec8e744cff450528ddec9769e5f3b557ca42bca47a"}
Jul 06 23:57:14.491680 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.491659    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm" event={"ID":"b7ac5bca-caa4-4fbc-bf90-de9897830455","Type":"ContainerStarted","Data":"b07527e0107c1dd4b636fb42567c9a2e7651dc561080b6719bcba5034ca306c1"}
Jul 06 23:57:14.491808 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.491702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm" event={"ID":"b7ac5bca-caa4-4fbc-bf90-de9897830455","Type":"ContainerStarted","Data":"5fb81f2abecd735717b6840cfdd37d89041312079a63b64224aaf1810fcdc0ae"}
Jul 06 23:57:14.491808 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.491743    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:14.508661 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.508615    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mjk9s" podStartSLOduration=16.730643905 podStartE2EDuration="23.508599884s" totalImagesPullingTime="6.777955979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:14.508339408 +0000 UTC m=+58.873643164" watchObservedRunningTime="2026-07-06 23:57:14.508599884 +0000 UTC m=+58.873903667"
Jul 06 23:57:14.530261 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:14.530212    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm" podStartSLOduration=1.530194044 podStartE2EDuration="1.530194044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:14.530020552 +0000 UTC m=+58.895324298" watchObservedRunningTime="2026-07-06 23:57:14.530194044 +0000 UTC m=+58.895497792"
Jul 06 23:57:15.386587 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:15.386561    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdsfs"
Jul 06 23:57:16.194697 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.194670    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"]
Jul 06 23:57:16.196936 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.196916    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:16.205006 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.204985    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9mmql\""
Jul 06 23:57:16.205374 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.205356    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 06 23:57:16.214379 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.214347    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"]
Jul 06 23:57:16.282786 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.282748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bdb4975e-e6af-4890-add8-22ef81d14bf6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jg8rv\" (UID: \"bdb4975e-e6af-4890-add8-22ef81d14bf6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:16.383471 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.383439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bdb4975e-e6af-4890-add8-22ef81d14bf6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jg8rv\" (UID: \"bdb4975e-e6af-4890-add8-22ef81d14bf6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:16.383658 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:16.383622    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 06 23:57:16.383740 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:16.383694    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb4975e-e6af-4890-add8-22ef81d14bf6-tls-certificates podName:bdb4975e-e6af-4890-add8-22ef81d14bf6 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:16.88367911 +0000 UTC m=+61.248982833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/bdb4975e-e6af-4890-add8-22ef81d14bf6-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-jg8rv" (UID: "bdb4975e-e6af-4890-add8-22ef81d14bf6") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 06 23:57:16.776752 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.776716    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-t2kph"]
Jul 06 23:57:16.779026 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.779009    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:16.781831 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.781806    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 23:57:16.781943 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.781830    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 23:57:16.782220 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.782197    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hfhbx\""
Jul 06 23:57:16.795889 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.795832    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-t2kph"]
Jul 06 23:57:16.888189 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.888147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bdb4975e-e6af-4890-add8-22ef81d14bf6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jg8rv\" (UID: \"bdb4975e-e6af-4890-add8-22ef81d14bf6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:16.888363 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.888214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfzd5\" (UniqueName: \"kubernetes.io/projected/e00e698a-1f4f-410b-9897-39b873210865-kube-api-access-nfzd5\") pod \"downloads-685bfb5f64-t2kph\" (UID: \"e00e698a-1f4f-410b-9897-39b873210865\") " pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:16.890697 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.890679    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bdb4975e-e6af-4890-add8-22ef81d14bf6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jg8rv\" (UID: \"bdb4975e-e6af-4890-add8-22ef81d14bf6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:16.989231 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.989105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfzd5\" (UniqueName: \"kubernetes.io/projected/e00e698a-1f4f-410b-9897-39b873210865-kube-api-access-nfzd5\") pod \"downloads-685bfb5f64-t2kph\" (UID: \"e00e698a-1f4f-410b-9897-39b873210865\") " pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:16.998098 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:16.998068    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfzd5\" (UniqueName: \"kubernetes.io/projected/e00e698a-1f4f-410b-9897-39b873210865-kube-api-access-nfzd5\") pod \"downloads-685bfb5f64-t2kph\" (UID: \"e00e698a-1f4f-410b-9897-39b873210865\") " pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:17.087813 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.087720    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:17.106888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.106864    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:17.225912 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.225883    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-t2kph"]
Jul 06 23:57:17.228802 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:17.228776    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00e698a_1f4f_410b_9897_39b873210865.slice/crio-0de90ca20aec9bad3bd4220444496ca3d71d81a3da1e65b6057504eb42c32098 WatchSource:0}: Error finding container 0de90ca20aec9bad3bd4220444496ca3d71d81a3da1e65b6057504eb42c32098: Status 404 returned error can't find the container with id 0de90ca20aec9bad3bd4220444496ca3d71d81a3da1e65b6057504eb42c32098
Jul 06 23:57:17.247267 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.247166    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"]
Jul 06 23:57:17.249520 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:17.249501    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb4975e_e6af_4890_add8_22ef81d14bf6.slice/crio-566a74443ed5d798eaeb147b550af51c89fff6735c50e1e1998fb612473c93ef WatchSource:0}: Error finding container 566a74443ed5d798eaeb147b550af51c89fff6735c50e1e1998fb612473c93ef: Status 404 returned error can't find the container with id 566a74443ed5d798eaeb147b550af51c89fff6735c50e1e1998fb612473c93ef
Jul 06 23:57:17.508095 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.508062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-t2kph" event={"ID":"e00e698a-1f4f-410b-9897-39b873210865","Type":"ContainerStarted","Data":"0de90ca20aec9bad3bd4220444496ca3d71d81a3da1e65b6057504eb42c32098"}
Jul 06 23:57:17.509225 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.509194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv" event={"ID":"bdb4975e-e6af-4890-add8-22ef81d14bf6","Type":"ContainerStarted","Data":"566a74443ed5d798eaeb147b550af51c89fff6735c50e1e1998fb612473c93ef"}
Jul 06 23:57:17.510385 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.510363    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6" event={"ID":"0e581d6c-c82e-442c-aee9-531aae945405","Type":"ContainerStarted","Data":"93a9088c5c3d440d934919b9d14459a3e3aef87ddbab2e650e7ffc29dac3674b"}
Jul 06 23:57:17.537600 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:17.537520    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7ccb4dd74b-gvmt6" podStartSLOduration=2.130172442 podStartE2EDuration="4.537503205s" totalImagesPullingTime="2.407330763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.535701178 +0000 UTC m=+61.901004923" watchObservedRunningTime="2026-07-06 23:57:17.537503205 +0000 UTC m=+61.902806950"
Jul 06 23:57:18.515731 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:18.515679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv" event={"ID":"bdb4975e-e6af-4890-add8-22ef81d14bf6","Type":"ContainerStarted","Data":"def5b0950569f8a776170d78390bc9b84b32aec9c7d5e1230f18c38e77df3672"}
Jul 06 23:57:18.516127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:18.515895    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:18.521879 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:18.521849    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv"
Jul 06 23:57:18.533384 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:18.533336    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jg8rv" podStartSLOduration=1.46466062 podStartE2EDuration="2.533316781s" totalImagesPullingTime="1.068656161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:18.532838784 +0000 UTC m=+62.898142542" watchObservedRunningTime="2026-07-06 23:57:18.533316781 +0000 UTC m=+62.898620528"
Jul 06 23:57:19.317012 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.316978    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"]
Jul 06 23:57:19.319998 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.319971    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.322671 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.322644    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k6gdr\""
Jul 06 23:57:19.323751 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.323726    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 06 23:57:19.323865 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.323734    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 23:57:19.323865 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.323736    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 23:57:19.324352 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.324327    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 06 23:57:19.324513 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.324497    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 23:57:19.343401 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.343372    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"]
Jul 06 23:57:19.411204 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.411174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f963865e-130e-4462-a2fc-060e814c82c5-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.411388 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.411212    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f963865e-130e-4462-a2fc-060e814c82c5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.411388 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.411323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtn69\" (UniqueName: \"kubernetes.io/projected/f963865e-130e-4462-a2fc-060e814c82c5-kube-api-access-wtn69\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.411388 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.411352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f963865e-130e-4462-a2fc-060e814c82c5-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.511871 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.511834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtn69\" (UniqueName: \"kubernetes.io/projected/f963865e-130e-4462-a2fc-060e814c82c5-kube-api-access-wtn69\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.511871 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.511878    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f963865e-130e-4462-a2fc-060e814c82c5-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.512089 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.511924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f963865e-130e-4462-a2fc-060e814c82c5-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.512089 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.511954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f963865e-130e-4462-a2fc-060e814c82c5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.512800 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.512767    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f963865e-130e-4462-a2fc-060e814c82c5-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.514791 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.514744    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f963865e-130e-4462-a2fc-060e814c82c5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.515331 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.515306    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f963865e-130e-4462-a2fc-060e814c82c5-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.522568 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.522543    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtn69\" (UniqueName: \"kubernetes.io/projected/f963865e-130e-4462-a2fc-060e814c82c5-kube-api-access-wtn69\") pod \"prometheus-operator-567f7b8b7c-q98k7\" (UID: \"f963865e-130e-4462-a2fc-060e814c82c5\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.631265 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.631186    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"
Jul 06 23:57:19.778260 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:19.778225    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7"]
Jul 06 23:57:19.781159 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:19.781130    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf963865e_130e_4462_a2fc_060e814c82c5.slice/crio-dd93a1ad50c0269a8f0fc2dad79e7d14622c3eccc0c82fdf5e0f8606b477dddb WatchSource:0}: Error finding container dd93a1ad50c0269a8f0fc2dad79e7d14622c3eccc0c82fdf5e0f8606b477dddb: Status 404 returned error can't find the container with id dd93a1ad50c0269a8f0fc2dad79e7d14622c3eccc0c82fdf5e0f8606b477dddb
Jul 06 23:57:20.524438 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:20.524364    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7" event={"ID":"f963865e-130e-4462-a2fc-060e814c82c5","Type":"ContainerStarted","Data":"dd93a1ad50c0269a8f0fc2dad79e7d14622c3eccc0c82fdf5e0f8606b477dddb"}
Jul 06 23:57:21.530432 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.530395    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7" event={"ID":"f963865e-130e-4462-a2fc-060e814c82c5","Type":"ContainerStarted","Data":"b53830572053a3fc4ef5c17d5cd2834fd15ba88bf3246bb4820794a0989b641d"}
Jul 06 23:57:21.530931 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.530450    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7" event={"ID":"f963865e-130e-4462-a2fc-060e814c82c5","Type":"ContainerStarted","Data":"b253e245369078c00e123f4de83a64bd647cda20c228a90ef37a7ddab35e47c6"}
Jul 06 23:57:21.552749 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.552677    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q98k7" podStartSLOduration=1.226352343 podStartE2EDuration="2.552656535s" totalImagesPullingTime="1.326304192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:21.552396092 +0000 UTC m=+65.917699841" watchObservedRunningTime="2026-07-06 23:57:21.552656535 +0000 UTC m=+65.917960281"
Jul 06 23:57:21.832609 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.832509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:57:21.832609 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.832604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:57:21.835837 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.835804    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:57:21.835962 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.835884    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:57:21.845672 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.845649    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da53e071-0353-4538-8837-240274613bc2-original-pull-secret\") pod \"global-pull-secret-syncer-zkcfs\" (UID: \"da53e071-0353-4538-8837-240274613bc2\") " pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:57:21.845774 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.845692    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448d3e76-6d59-46b3-af7d-7714f0fd3772-metrics-certs\") pod \"network-metrics-daemon-fwwp7\" (UID: \"448d3e76-6d59-46b3-af7d-7714f0fd3772\") " pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:57:21.933021 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.932982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:57:21.936072 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.936034    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:57:21.946948 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.946923    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:57:21.957666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:21.957637    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hrc5\" (UniqueName: \"kubernetes.io/projected/37d6930f-b6fa-4f56-b860-ab8fdb517d76-kube-api-access-7hrc5\") pod \"network-check-target-6wfrf\" (UID: \"37d6930f-b6fa-4f56-b860-ab8fdb517d76\") " pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:57:22.063971 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.063940    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6v6rl\""
Jul 06 23:57:22.066199 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.066176    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkcfs"
Jul 06 23:57:22.071331 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.071304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwwp7"
Jul 06 23:57:22.074156 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.074122    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m54k2\""
Jul 06 23:57:22.082407 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.082377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:57:22.255281 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.255223    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zkcfs"]
Jul 06 23:57:22.285783 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.285739    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fwwp7"]
Jul 06 23:57:22.288760 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:22.288730    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448d3e76_6d59_46b3_af7d_7714f0fd3772.slice/crio-4edf00d3bce833b99c5a56f04c2bf2f398a60a9e53e27f0a1efd456338752e49 WatchSource:0}: Error finding container 4edf00d3bce833b99c5a56f04c2bf2f398a60a9e53e27f0a1efd456338752e49: Status 404 returned error can't find the container with id 4edf00d3bce833b99c5a56f04c2bf2f398a60a9e53e27f0a1efd456338752e49
Jul 06 23:57:22.315775 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.315739    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6wfrf"]
Jul 06 23:57:22.318898 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:22.318871    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d6930f_b6fa_4f56_b860_ab8fdb517d76.slice/crio-5c692fe8051dcb40ec69a9dd6b4b395b333487d0414054fdfb78e2363aa6a0b2 WatchSource:0}: Error finding container 5c692fe8051dcb40ec69a9dd6b4b395b333487d0414054fdfb78e2363aa6a0b2: Status 404 returned error can't find the container with id 5c692fe8051dcb40ec69a9dd6b4b395b333487d0414054fdfb78e2363aa6a0b2
Jul 06 23:57:22.535124 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.535084    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fwwp7" event={"ID":"448d3e76-6d59-46b3-af7d-7714f0fd3772","Type":"ContainerStarted","Data":"4edf00d3bce833b99c5a56f04c2bf2f398a60a9e53e27f0a1efd456338752e49"}
Jul 06 23:57:22.536357 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.536327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zkcfs" event={"ID":"da53e071-0353-4538-8837-240274613bc2","Type":"ContainerStarted","Data":"8c3148d03cf66ecdfc93850ed20a1bb3feb1aaccdd67dfb9a8168205355fdfec"}
Jul 06 23:57:22.537548 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:22.537504    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6wfrf" event={"ID":"37d6930f-b6fa-4f56-b860-ab8fdb517d76","Type":"ContainerStarted","Data":"5c692fe8051dcb40ec69a9dd6b4b395b333487d0414054fdfb78e2363aa6a0b2"}
Jul 06 23:57:24.065402 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.065367    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"]
Jul 06 23:57:24.068549 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.068503    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.079807 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.079122    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 06 23:57:24.082241 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.082223    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rj7nd\""
Jul 06 23:57:24.085972 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.082895    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 06 23:57:24.085972 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.082970    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 06 23:57:24.085972 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.084614    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dmkxm"]
Jul 06 23:57:24.087930 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.087410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.095345 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.092640    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ptknx\""
Jul 06 23:57:24.095345 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.092904    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 23:57:24.095345 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.093100    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 23:57:24.095345 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.094495    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 23:57:24.095345 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.095272    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"]
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168235    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9jjv\" (UniqueName: \"kubernetes.io/projected/2faf3977-df0c-4d32-af9e-1a25cd8a3996-kube-api-access-s9jjv\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168262    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-root\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168319    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39ec36b4-f2f4-4895-bf89-c7f688d56b14-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-textfile\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-tls\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2faf3977-df0c-4d32-af9e-1a25cd8a3996-metrics-client-ca\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168509    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39ec36b4-f2f4-4895-bf89-c7f688d56b14-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dfqk\" (UniqueName: \"kubernetes.io/projected/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-api-access-5dfqk\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168589    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-sys\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168610    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-wtmp\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.168825 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.168642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-accelerators-collector-config\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-textfile\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269383    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-tls\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269410    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2faf3977-df0c-4d32-af9e-1a25cd8a3996-metrics-client-ca\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269436    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39ec36b4-f2f4-4895-bf89-c7f688d56b14-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269551    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dfqk\" (UniqueName: \"kubernetes.io/projected/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-api-access-5dfqk\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269582    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-sys\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-wtmp\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-accelerators-collector-config\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9jjv\" (UniqueName: \"kubernetes.io/projected/2faf3977-df0c-4d32-af9e-1a25cd8a3996-kube-api-access-s9jjv\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269728    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-root\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269758    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.269787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39ec36b4-f2f4-4895-bf89-c7f688d56b14-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.271734 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.270257    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39ec36b4-f2f4-4895-bf89-c7f688d56b14-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.270563    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-textfile\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.270625    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-sys\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.270700    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-wtmp\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.271237    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-accelerators-collector-config\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.271299    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2faf3977-df0c-4d32-af9e-1a25cd8a3996-root\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:24.271372    2583 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:24.271427    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-tls podName:39ec36b4-f2f4-4895-bf89-c7f688d56b14 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:24.771408198 +0000 UTC m=+69.136711923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-vrbgs" (UID: "39ec36b4-f2f4-4895-bf89-c7f688d56b14") : secret "kube-state-metrics-tls" not found
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.271587    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2faf3977-df0c-4d32-af9e-1a25cd8a3996-metrics-client-ca\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.272093    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39ec36b4-f2f4-4895-bf89-c7f688d56b14-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.272804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.272657    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.275689 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.275657    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.275814 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.275751    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.278931 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.278907    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2faf3977-df0c-4d32-af9e-1a25cd8a3996-node-exporter-tls\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.321817 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.321788    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9jjv\" (UniqueName: \"kubernetes.io/projected/2faf3977-df0c-4d32-af9e-1a25cd8a3996-kube-api-access-s9jjv\") pod \"node-exporter-dmkxm\" (UID: \"2faf3977-df0c-4d32-af9e-1a25cd8a3996\") " pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.325380 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.325352    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dfqk\" (UniqueName: \"kubernetes.io/projected/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-api-access-5dfqk\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.406512 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.406426    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dmkxm"
Jul 06 23:57:24.418378 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:24.418329    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2faf3977_df0c_4d32_af9e_1a25cd8a3996.slice/crio-6e89af7b470764a6c6dacbe9296262d580671da6d670c8c514a33157582c84b0 WatchSource:0}: Error finding container 6e89af7b470764a6c6dacbe9296262d580671da6d670c8c514a33157582c84b0: Status 404 returned error can't find the container with id 6e89af7b470764a6c6dacbe9296262d580671da6d670c8c514a33157582c84b0
Jul 06 23:57:24.555058 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.555018    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fwwp7" event={"ID":"448d3e76-6d59-46b3-af7d-7714f0fd3772","Type":"ContainerStarted","Data":"474dfa4f89b21a0ec4324c008d01615342ccad6956a245a6070e92dac06f0d99"}
Jul 06 23:57:24.555058 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.555091    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fwwp7" event={"ID":"448d3e76-6d59-46b3-af7d-7714f0fd3772","Type":"ContainerStarted","Data":"7234b714474d14f0830877c0dc85279ef3b06240e8b4d3b13262efb6bf58f962"}
Jul 06 23:57:24.557219 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.557188    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dmkxm" event={"ID":"2faf3977-df0c-4d32-af9e-1a25cd8a3996","Type":"ContainerStarted","Data":"6e89af7b470764a6c6dacbe9296262d580671da6d670c8c514a33157582c84b0"}
Jul 06 23:57:24.601205 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.601066    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fwwp7" podStartSLOduration=67.302418448 podStartE2EDuration="1m8.601042504s" totalImagesPullingTime="1.298624056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:24.590163117 +0000 UTC m=+68.955466864" watchObservedRunningTime="2026-07-06 23:57:24.601042504 +0000 UTC m=+68.966346251"
Jul 06 23:57:24.774893 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.774856    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.778079 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.778049    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39ec36b4-f2f4-4895-bf89-c7f688d56b14-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vrbgs\" (UID: \"39ec36b4-f2f4-4895-bf89-c7f688d56b14\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:24.857478 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.857446    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:57:24.861021 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.860991    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.864109    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.864305    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.864748    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.864998    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.865173    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.865384    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.865580    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.865798    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8bxds\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.865988    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 23:57:24.867738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.866171    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 23:57:24.887463 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.887435    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:57:24.978708 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978668    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-web-config\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.978897 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.978897 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978764    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.978897 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978796    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.978897 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.978897 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978890    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979215 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.978987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979215 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.979043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979215 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.979075    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-846wf\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-kube-api-access-846wf\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979215 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.979109    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-out\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979215 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.979133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979592 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.979217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.979592 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.979264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:24.983222 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:24.983193    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"
Jul 06 23:57:25.080823 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846wf\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-kube-api-access-846wf\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-out\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.080988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.081024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.081052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.081092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-web-config\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.081120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.081152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081275 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.081180    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.081861 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:25.081341    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle podName:fb3680e4-50ef-47b7-8743-aa8dd81a8585 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:25.58131785 +0000 UTC m=+69.946621596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585") : configmap references non-existent config key: ca-bundle.crt
Jul 06 23:57:25.092580 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.083288    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.092580 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.084810    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.092580 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:25.084904    2583 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 06 23:57:25.092580 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:57:25.084967    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls podName:fb3680e4-50ef-47b7-8743-aa8dd81a8585 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:25.584948549 +0000 UTC m=+69.950252282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585") : secret "alertmanager-main-tls" not found
Jul 06 23:57:25.103549 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.100510    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-out\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.107056 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.107016    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.108508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.107968    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-web-config\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.110184 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.110108    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.110845 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.110791    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.112000 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.111956    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-846wf\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-kube-api-access-846wf\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.113123 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.113099    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.117386 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.117326    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.126481 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.120690    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.218878 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.218751    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs"]
Jul 06 23:57:25.226501 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:25.226467    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ec36b4_f2f4_4895_bf89_c7f688d56b14.slice/crio-176c6b900fa08ca5021b6793e161cd0737a50814f25b671b8e2eff66592243f7 WatchSource:0}: Error finding container 176c6b900fa08ca5021b6793e161cd0737a50814f25b671b8e2eff66592243f7: Status 404 returned error can't find the container with id 176c6b900fa08ca5021b6793e161cd0737a50814f25b671b8e2eff66592243f7
Jul 06 23:57:25.564662 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.564622    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs" event={"ID":"39ec36b4-f2f4-4895-bf89-c7f688d56b14","Type":"ContainerStarted","Data":"176c6b900fa08ca5021b6793e161cd0737a50814f25b671b8e2eff66592243f7"}
Jul 06 23:57:25.586223 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.586184    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.586400 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.586264    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.587740 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.587663    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.590244 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.590216    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:25.781351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:25.781315    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:28.532816 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.532785    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"]
Jul 06 23:57:28.550461 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.549755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"]
Jul 06 23:57:28.550461 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.549857    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.554882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.554859    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jh4nz\""
Jul 06 23:57:28.556151 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.555938    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 06 23:57:28.617684 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.617636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6bced467-26ca-441e-b1bd-fe12027bf6be-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-c5x2n\" (UID: \"6bced467-26ca-441e-b1bd-fe12027bf6be\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.617684 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.617685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2c5j\" (UniqueName: \"kubernetes.io/projected/6bced467-26ca-441e-b1bd-fe12027bf6be-kube-api-access-d2c5j\") pod \"monitoring-plugin-d5dcff7cf-c5x2n\" (UID: \"6bced467-26ca-441e-b1bd-fe12027bf6be\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.718942 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.718902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2c5j\" (UniqueName: \"kubernetes.io/projected/6bced467-26ca-441e-b1bd-fe12027bf6be-kube-api-access-d2c5j\") pod \"monitoring-plugin-d5dcff7cf-c5x2n\" (UID: \"6bced467-26ca-441e-b1bd-fe12027bf6be\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.719114 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.719004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6bced467-26ca-441e-b1bd-fe12027bf6be-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-c5x2n\" (UID: \"6bced467-26ca-441e-b1bd-fe12027bf6be\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.722127 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.722079    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6bced467-26ca-441e-b1bd-fe12027bf6be-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-c5x2n\" (UID: \"6bced467-26ca-441e-b1bd-fe12027bf6be\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.739159 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.739105    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2c5j\" (UniqueName: \"kubernetes.io/projected/6bced467-26ca-441e-b1bd-fe12027bf6be-kube-api-access-d2c5j\") pod \"monitoring-plugin-d5dcff7cf-c5x2n\" (UID: \"6bced467-26ca-441e-b1bd-fe12027bf6be\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:28.862286 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:28.862208    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:29.295087 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.295057    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-68fff647f-jw9jx"]
Jul 06 23:57:29.325883 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.325852    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68fff647f-jw9jx"]
Jul 06 23:57:29.326079 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.326001    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.329046 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.328719    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 06 23:57:29.329046 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.328763    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 06 23:57:29.329046 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.328771    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vq8v8\""
Jul 06 23:57:29.329046 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.328796    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 06 23:57:29.329046 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.328809    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 06 23:57:29.329390 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.329141    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 06 23:57:29.334302 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.334145    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 06 23:57:29.426018 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.425988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-serving-certs-ca-bundle\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426018 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-secret-telemeter-client\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426248 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-telemeter-client-tls\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426248 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426124    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426248 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426176    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426401 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426253    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-metrics-client-ca\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426401 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426275    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-federate-client-tls\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.426401 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.426292    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q695p\" (UniqueName: \"kubernetes.io/projected/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-kube-api-access-q695p\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.527301 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-serving-certs-ca-bundle\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.527301 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-secret-telemeter-client\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-telemeter-client-tls\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527377    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527416    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-metrics-client-ca\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-federate-client-tls\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.527546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q695p\" (UniqueName: \"kubernetes.io/projected/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-kube-api-access-q695p\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.528091    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-serving-certs-ca-bundle\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.528499    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-metrics-client-ca\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.528737 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.528636    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.530350 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.530327    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-secret-telemeter-client\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.530546 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.530503    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-telemeter-client-tls\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.530705 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.530684    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-federate-client-tls\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.530762 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.530684    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.537174 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.537153    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q695p\" (UniqueName: \"kubernetes.io/projected/5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee-kube-api-access-q695p\") pod \"telemeter-client-68fff647f-jw9jx\" (UID: \"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee\") " pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:29.637446 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:29.637369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx"
Jul 06 23:57:30.135900 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.135865    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:57:30.139867 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.139839    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.145472 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.145310    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b4su5h3stafmr\""
Jul 06 23:57:30.145472 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.145328    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 23:57:30.145472 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.145374    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 23:57:30.145472 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.145374    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 23:57:30.145750 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.145636    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 23:57:30.145850 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.145835    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 23:57:30.146032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.146014    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9zxpb\""
Jul 06 23:57:30.146474 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.146296    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 23:57:30.146618 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.146552    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 23:57:30.147317 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.147294    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 23:57:30.147576 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.147556    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 23:57:30.148447 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.147750    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 23:57:30.148447 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.147803    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 23:57:30.148447 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.147858    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 23:57:30.155745 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.155725    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 23:57:30.159306 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.159282    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 23:57:30.171447 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.171413    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 23:57:30.177078 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.176983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:57:30.234591 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.234793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234612    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.234793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.234793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.234793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.234793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.234793 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234803    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234857    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-config\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234882    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v45bd\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-kube-api-access-v45bd\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234954    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.234977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.235008    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.235036    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.235066    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.235095    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235500 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.235144    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.235500 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.235170    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.335985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336240 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336055    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336240 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336240 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336240 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336240 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336240 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-config\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v45bd\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-kube-api-access-v45bd\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336376    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336431    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.336488 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.337121 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336500    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.337121 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.337121 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.336573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.337281 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.337245    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.337569 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.337521    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.338780 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.338409    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.339755 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.339594    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.339965 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.339877    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.340199 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.340089    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.340471 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.340449    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.341477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.341105    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.341477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.341158    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.341477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.341454    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.341710 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.341566    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.341911 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.341871    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.342597 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.342570    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.342959 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.342913    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.343781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.343741    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.343781 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.343761    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.344790 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.344724    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.344790 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.344755    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.344924 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.344905    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-config\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.348984 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.348937    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v45bd\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-kube-api-access-v45bd\") pod \"prometheus-k8s-0\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:30.455444 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:30.455370    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:35.501081 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:35.501048    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64f66b6665-lpmjm"
Jul 06 23:57:36.700097 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:36.699837    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"]
Jul 06 23:57:36.867444 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:36.867408    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bced467_26ca_441e_b1bd_fe12027bf6be.slice/crio-4a64383d91f5e7f51f67432ecc85f6d8e709408783c33d11ffddaf2a5a90298e WatchSource:0}: Error finding container 4a64383d91f5e7f51f67432ecc85f6d8e709408783c33d11ffddaf2a5a90298e: Status 404 returned error can't find the container with id 4a64383d91f5e7f51f67432ecc85f6d8e709408783c33d11ffddaf2a5a90298e
Jul 06 23:57:36.953324 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:36.953231    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68fff647f-jw9jx"]
Jul 06 23:57:36.958783 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:36.958750    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:57:36.963273 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:36.963247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:57:37.235028 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:37.234977    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937977c8_f6de_47f3_a6f9_885f4f5083a1.slice/crio-f6976a924bd77b0813d1628965a8898c001282c71c514e49c5a9f552d8d51fcb WatchSource:0}: Error finding container f6976a924bd77b0813d1628965a8898c001282c71c514e49c5a9f552d8d51fcb: Status 404 returned error can't find the container with id f6976a924bd77b0813d1628965a8898c001282c71c514e49c5a9f552d8d51fcb
Jul 06 23:57:37.235897 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:37.235866    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3680e4_50ef_47b7_8743_aa8dd81a8585.slice/crio-e769a778ec1e6360a32865b0b726ca23d28810ea37ac20ac924a5d978e96cfb9 WatchSource:0}: Error finding container e769a778ec1e6360a32865b0b726ca23d28810ea37ac20ac924a5d978e96cfb9: Status 404 returned error can't find the container with id e769a778ec1e6360a32865b0b726ca23d28810ea37ac20ac924a5d978e96cfb9
Jul 06 23:57:37.236697 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:57:37.236584    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffaf726_bc0a_4db6_bd89_b2c4fffc32ee.slice/crio-9360ce8856108ed6bcea7a73534d8ea0443654c1b135e6a34f72965a92e4c56b WatchSource:0}: Error finding container 9360ce8856108ed6bcea7a73534d8ea0443654c1b135e6a34f72965a92e4c56b: Status 404 returned error can't find the container with id 9360ce8856108ed6bcea7a73534d8ea0443654c1b135e6a34f72965a92e4c56b
Jul 06 23:57:37.616738 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.616562    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-t2kph" event={"ID":"e00e698a-1f4f-410b-9897-39b873210865","Type":"ContainerStarted","Data":"b942b7a046754f58902d42680c4bd5aa70ad4ae4211794f2ab0e5999a1552019"}
Jul 06 23:57:37.616891 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.616867    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:37.618331 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.618285    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zkcfs" event={"ID":"da53e071-0353-4538-8837-240274613bc2","Type":"ContainerStarted","Data":"c2f91c2e590d816a9f0fea86d0d067665943009cec452e3c8113012cf498aa4c"}
Jul 06 23:57:37.618463 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.618343    2583 patch_prober.go:28] interesting pod/downloads-685bfb5f64-t2kph container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.13:8080/\": dial tcp 10.134.0.13:8080: connect: connection refused" start-of-body=
Jul 06 23:57:37.618463 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.618384    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-t2kph" podUID="e00e698a-1f4f-410b-9897-39b873210865" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.13:8080/\": dial tcp 10.134.0.13:8080: connect: connection refused"
Jul 06 23:57:37.619736 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.619667    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"f6976a924bd77b0813d1628965a8898c001282c71c514e49c5a9f552d8d51fcb"}
Jul 06 23:57:37.622277 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.622256    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6wfrf" event={"ID":"37d6930f-b6fa-4f56-b860-ab8fdb517d76","Type":"ContainerStarted","Data":"a934701698713304a1d99f137b45d998ecf197a46a26d19ffa2b501da79b2665"}
Jul 06 23:57:37.624456 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.624436    2583 generic.go:356] "Generic (PLEG): container finished" podID="2faf3977-df0c-4d32-af9e-1a25cd8a3996" containerID="e2a662ab130a50f0f0387a7ee5c3de41faeb225fe5e686a6b93d4954417f2971" exitCode=0
Jul 06 23:57:37.624570 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.624507    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dmkxm" event={"ID":"2faf3977-df0c-4d32-af9e-1a25cd8a3996","Type":"ContainerDied","Data":"e2a662ab130a50f0f0387a7ee5c3de41faeb225fe5e686a6b93d4954417f2971"}
Jul 06 23:57:37.625815 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.625788    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"e769a778ec1e6360a32865b0b726ca23d28810ea37ac20ac924a5d978e96cfb9"}
Jul 06 23:57:37.627971 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.627461    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n" event={"ID":"6bced467-26ca-441e-b1bd-fe12027bf6be","Type":"ContainerStarted","Data":"4a64383d91f5e7f51f67432ecc85f6d8e709408783c33d11ffddaf2a5a90298e"}
Jul 06 23:57:37.629303 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.629276    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx" event={"ID":"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee","Type":"ContainerStarted","Data":"9360ce8856108ed6bcea7a73534d8ea0443654c1b135e6a34f72965a92e4c56b"}
Jul 06 23:57:37.631430 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.631409    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs" event={"ID":"39ec36b4-f2f4-4895-bf89-c7f688d56b14","Type":"ContainerStarted","Data":"376e2392faf68ff380c5793f39a58987d741527bc8a8a8cc27c7b66f321556de"}
Jul 06 23:57:37.632160 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.632118    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:57:37.637794 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.637738    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-t2kph" podStartSLOduration=1.6310939850000001 podStartE2EDuration="21.637718517s" totalImagesPullingTime="20.006624532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.636992524 +0000 UTC m=+82.002296267" watchObservedRunningTime="2026-07-06 23:57:37.637718517 +0000 UTC m=+82.003022286"
Jul 06 23:57:37.657698 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.657642    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6wfrf" podStartSLOduration=67.473186815 podStartE2EDuration="1m21.657625915s" totalImagesPullingTime="14.1844391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.656961643 +0000 UTC m=+82.022265390" watchObservedRunningTime="2026-07-06 23:57:37.657625915 +0000 UTC m=+82.022929663"
Jul 06 23:57:37.705307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:37.705240    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zkcfs" podStartSLOduration=67.456309438 podStartE2EDuration="1m21.705219946s" totalImagesPullingTime="14.248910508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.704454082 +0000 UTC m=+82.069757828" watchObservedRunningTime="2026-07-06 23:57:37.705219946 +0000 UTC m=+82.070523690"
Jul 06 23:57:38.643605 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.641606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dmkxm" event={"ID":"2faf3977-df0c-4d32-af9e-1a25cd8a3996","Type":"ContainerStarted","Data":"55dceb4dbaf2e8cc3c7d9ef523b7d3b2c2f13e03dd804bb3f4d7597323b8e43d"}
Jul 06 23:57:38.643605 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.641678    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dmkxm" event={"ID":"2faf3977-df0c-4d32-af9e-1a25cd8a3996","Type":"ContainerStarted","Data":"e25ff2b73aa639659cbd04d5fd187843ddc9b1804140c65b569816ec7a39cf54"}
Jul 06 23:57:38.660941 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.659948    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs" event={"ID":"39ec36b4-f2f4-4895-bf89-c7f688d56b14","Type":"ContainerStarted","Data":"d8baab393e3fea6268a14c7ba35919c85606706160add10f2d9c505771556276"}
Jul 06 23:57:38.660941 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.660023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs" event={"ID":"39ec36b4-f2f4-4895-bf89-c7f688d56b14","Type":"ContainerStarted","Data":"71fdfee9f45cbafdca5b316468da0d3a46b9da5f37fab40c6c5c47fa32956597"}
Jul 06 23:57:38.671596 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.669516    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-t2kph"
Jul 06 23:57:38.678227 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.677435    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dmkxm" podStartSLOduration=2.49108487 podStartE2EDuration="14.677412659s" totalImagesPullingTime="12.076040392s" totalInitContainerRuntime="110.287397ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:38.677127687 +0000 UTC m=+83.042431457" watchObservedRunningTime="2026-07-06 23:57:38.677412659 +0000 UTC m=+83.042716407"
Jul 06 23:57:38.698937 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:38.698880    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vrbgs" podStartSLOduration=2.420514776 podStartE2EDuration="14.698856476s" totalImagesPullingTime="12.2783417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:38.69802211 +0000 UTC m=+83.063325870" watchObservedRunningTime="2026-07-06 23:57:38.698856476 +0000 UTC m=+83.064160225"
Jul 06 23:57:41.688423 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.688383    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac" exitCode=0
Jul 06 23:57:41.688919 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.688465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"}
Jul 06 23:57:41.691101 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.691077    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839" exitCode=0
Jul 06 23:57:41.691212 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.691165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839"}
Jul 06 23:57:41.694805 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.694777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n" event={"ID":"6bced467-26ca-441e-b1bd-fe12027bf6be","Type":"ContainerStarted","Data":"00d8aabfdc70fdbc30c12fb26b20edb5e53391bbb443336bbd967c22199be1d7"}
Jul 06 23:57:41.695605 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.695582    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:41.697483 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.697453    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx" event={"ID":"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee","Type":"ContainerStarted","Data":"fc64926f0fd7e60f9f4f23e81cc467269064bb878a65c78416c9de557d297255"}
Jul 06 23:57:41.701404 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:41.701379    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n"
Jul 06 23:57:42.713789 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:42.713589    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx" event={"ID":"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee","Type":"ContainerStarted","Data":"41aed9fb51b05c7dcc118c493c68ca1dbd7fccb2fb54750d2c6f24cf0e2e64c8"}
Jul 06 23:57:42.713789 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:42.713704    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx" event={"ID":"5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee","Type":"ContainerStarted","Data":"0b09f7796614f2366aa00792fcc9f5c8bbacca90f12b0706ec3f7beb3e164c42"}
Jul 06 23:57:42.745807 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:42.744366    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-68fff647f-jw9jx" podStartSLOduration=9.55088219 podStartE2EDuration="13.744341593s" totalImagesPullingTime="4.193459403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:42.742304538 +0000 UTC m=+87.107608285" watchObservedRunningTime="2026-07-06 23:57:42.744341593 +0000 UTC m=+87.109645341"
Jul 06 23:57:42.745807 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:42.745250    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c5x2n" podStartSLOduration=10.188923097 podStartE2EDuration="14.745236809s" totalImagesPullingTime="4.556313712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:41.76441969 +0000 UTC m=+86.129723436" watchObservedRunningTime="2026-07-06 23:57:42.745236809 +0000 UTC m=+87.110540556"
Jul 06 23:57:44.727766 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:44.727571    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a"}
Jul 06 23:57:44.727766 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:44.727663    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5"}
Jul 06 23:57:46.740340 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:46.740303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"}
Jul 06 23:57:46.740809 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:46.740408    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"}
Jul 06 23:57:46.743821 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:46.743797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb"}
Jul 06 23:57:46.743933 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:46.743870    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052"}
Jul 06 23:57:46.743933 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:46.743913    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576"}
Jul 06 23:57:47.751979 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:47.751939    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerStarted","Data":"27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4"}
Jul 06 23:57:47.786065 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:47.786008    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=16.10859192 podStartE2EDuration="23.785989883s" totalImagesPullingTime="7.654116392s" totalInitContainerRuntime="23.281571ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:47.78429265 +0000 UTC m=+92.149596397" watchObservedRunningTime="2026-07-06 23:57:47.785989883 +0000 UTC m=+92.151293628"
Jul 06 23:57:48.759383 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:48.759345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"}
Jul 06 23:57:48.759782 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:48.759468    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"}
Jul 06 23:57:49.766756 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:49.766721    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"}
Jul 06 23:57:49.767195 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:49.766787    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerStarted","Data":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"}
Jul 06 23:57:49.798009 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:49.797959    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=9.036414871 podStartE2EDuration="19.797938685s" totalImagesPullingTime="10.737742493s" totalInitContainerRuntime="23.781321ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:49.795992541 +0000 UTC m=+94.161296305" watchObservedRunningTime="2026-07-06 23:57:49.797938685 +0000 UTC m=+94.163242431"
Jul 06 23:57:50.456583 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:57:50.456550    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:08.663625 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:08.663592    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6wfrf"
Jul 06 23:58:30.456198 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:30.456163    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:30.518485 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:30.518453    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:30.955212 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:30.955188    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:44.265663 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.265634    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:44.266136 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.265910    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="alertmanager" containerID="cri-o://5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5" gracePeriod=120
Jul 06 23:58:44.266136 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.265999    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-metric" containerID="cri-o://4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb" gracePeriod=120
Jul 06 23:58:44.266136 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.266022    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-web" containerID="cri-o://df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576" gracePeriod=120
Jul 06 23:58:44.266136 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.266038    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="config-reloader" containerID="cri-o://ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a" gracePeriod=120
Jul 06 23:58:44.266136 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.266085    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy" containerID="cri-o://6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052" gracePeriod=120
Jul 06 23:58:44.266405 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.266145    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="prom-label-proxy" containerID="cri-o://27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4" gracePeriod=120
Jul 06 23:58:44.995096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995064    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4" exitCode=0
Jul 06 23:58:44.995096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995092    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb" exitCode=0
Jul 06 23:58:44.995096 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995101    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052" exitCode=0
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995107    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a" exitCode=0
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995112    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5" exitCode=0
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995133    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4"}
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb"}
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995226    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052"}
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a"}
Jul 06 23:58:44.995320 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:44.995274    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5"}
Jul 06 23:58:45.515916 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.515896    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:45.628171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628086    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-metrics-client-ca\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628126    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-main-db\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628152    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628171 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628170    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-tls-assets\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628333    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-web-config\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628379    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-web\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628477 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628438    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:45.628666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628449    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628519    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-metric\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628516    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:45.628666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628584    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-846wf\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-kube-api-access-846wf\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628620    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-volume\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628645    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-out\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628987 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628728    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628987 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.628757    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-cluster-tls-config\") pod \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\" (UID: \"fb3680e4-50ef-47b7-8743-aa8dd81a8585\") "
Jul 06 23:58:45.628987 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.629014    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-metrics-client-ca\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.629237 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.629039    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-main-db\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.629580 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.629551    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:45.631948 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.631900    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.632298 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.632271    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.632383 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.632351    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:45.632517 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.632492    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-out" (OuterVolumeSpecName: "config-out") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:45.632666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.632642    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-kube-api-access-846wf" (OuterVolumeSpecName: "kube-api-access-846wf") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "kube-api-access-846wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:45.632823 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.632801    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.632985 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.632969    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.633292 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.633260    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-volume" (OuterVolumeSpecName: "config-volume") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.636591 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.636504    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.642642 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.642618    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-web-config" (OuterVolumeSpecName: "web-config") pod "fb3680e4-50ef-47b7-8743-aa8dd81a8585" (UID: "fb3680e4-50ef-47b7-8743-aa8dd81a8585"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:45.729653 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729618    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-main-tls\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729653 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729646    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-tls-assets\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729653 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729655    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-web-config\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729665    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729674    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729683    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729693    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-846wf\" (UniqueName: \"kubernetes.io/projected/fb3680e4-50ef-47b7-8743-aa8dd81a8585-kube-api-access-846wf\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729700    2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-volume\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729708    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb3680e4-50ef-47b7-8743-aa8dd81a8585-config-out\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729716    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3680e4-50ef-47b7-8743-aa8dd81a8585-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:45.729863 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:45.729723    2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb3680e4-50ef-47b7-8743-aa8dd81a8585-cluster-tls-config\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:46.001767 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.001731    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerID="df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576" exitCode=0
Jul 06 23:58:46.001917 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.001795    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576"}
Jul 06 23:58:46.001917 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.001828    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.001917 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.001868    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb3680e4-50ef-47b7-8743-aa8dd81a8585","Type":"ContainerDied","Data":"e769a778ec1e6360a32865b0b726ca23d28810ea37ac20ac924a5d978e96cfb9"}
Jul 06 23:58:46.001917 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.001916    2583 scope.go:117] "RemoveContainer" containerID="27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4"
Jul 06 23:58:46.011940 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.011918    2583 scope.go:117] "RemoveContainer" containerID="4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb"
Jul 06 23:58:46.018828 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.018811    2583 scope.go:117] "RemoveContainer" containerID="6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052"
Jul 06 23:58:46.025187 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.025172    2583 scope.go:117] "RemoveContainer" containerID="df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576"
Jul 06 23:58:46.028835 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.028816    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:46.032620 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.032601    2583 scope.go:117] "RemoveContainer" containerID="ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a"
Jul 06 23:58:46.036581 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.036560    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:46.040040 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.040023    2583 scope.go:117] "RemoveContainer" containerID="5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5"
Jul 06 23:58:46.046805 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.046789    2583 scope.go:117] "RemoveContainer" containerID="6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839"
Jul 06 23:58:46.053431 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.053402    2583 scope.go:117] "RemoveContainer" containerID="27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4"
Jul 06 23:58:46.053709 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.053689    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4\": container with ID starting with 27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4 not found: ID does not exist" containerID="27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4"
Jul 06 23:58:46.053788 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.053720    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4"} err="failed to get container status \"27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4\": rpc error: code = NotFound desc = could not find container \"27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4\": container with ID starting with 27ac111c2c330e7e36cffab4729890116fae0a925befc4fba3925e5c64d9d8c4 not found: ID does not exist"
Jul 06 23:58:46.053788 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.053780    2583 scope.go:117] "RemoveContainer" containerID="4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb"
Jul 06 23:58:46.054040 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.054023    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb\": container with ID starting with 4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb not found: ID does not exist" containerID="4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb"
Jul 06 23:58:46.054103 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054049    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb"} err="failed to get container status \"4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb\": rpc error: code = NotFound desc = could not find container \"4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb\": container with ID starting with 4c9c938d7bd915267398f4480d980fa46285910eb64a520620a7c7810a67a5eb not found: ID does not exist"
Jul 06 23:58:46.054103 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054083    2583 scope.go:117] "RemoveContainer" containerID="6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052"
Jul 06 23:58:46.054318 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.054301    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052\": container with ID starting with 6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052 not found: ID does not exist" containerID="6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052"
Jul 06 23:58:46.054364 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054323    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052"} err="failed to get container status \"6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052\": rpc error: code = NotFound desc = could not find container \"6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052\": container with ID starting with 6fb16b08536b61e1325c0e40af8939d10781a8bb24e3fd33babc6d59bc969052 not found: ID does not exist"
Jul 06 23:58:46.054364 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054337    2583 scope.go:117] "RemoveContainer" containerID="df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576"
Jul 06 23:58:46.054605 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.054584    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576\": container with ID starting with df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576 not found: ID does not exist" containerID="df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576"
Jul 06 23:58:46.054660 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054610    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576"} err="failed to get container status \"df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576\": rpc error: code = NotFound desc = could not find container \"df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576\": container with ID starting with df88269911d1e4dd033a21d16a78e6aa56f16e24cc80b36574ff70789b9d6576 not found: ID does not exist"
Jul 06 23:58:46.054660 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054626    2583 scope.go:117] "RemoveContainer" containerID="ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a"
Jul 06 23:58:46.054870 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.054852    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a\": container with ID starting with ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a not found: ID does not exist" containerID="ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a"
Jul 06 23:58:46.054909 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054880    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a"} err="failed to get container status \"ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a\": rpc error: code = NotFound desc = could not find container \"ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a\": container with ID starting with ff83e77609e16582e0ce99febd29e831af48f74fac028cfaf1b7e6307a34745a not found: ID does not exist"
Jul 06 23:58:46.054909 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.054894    2583 scope.go:117] "RemoveContainer" containerID="5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5"
Jul 06 23:58:46.055125 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.055110    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5\": container with ID starting with 5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5 not found: ID does not exist" containerID="5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5"
Jul 06 23:58:46.055179 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.055128    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5"} err="failed to get container status \"5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5\": rpc error: code = NotFound desc = could not find container \"5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5\": container with ID starting with 5ff904b29f9855ad31ac77fe4b0f4f0fa905f01113b8701b73a7f86e9cbcc7b5 not found: ID does not exist"
Jul 06 23:58:46.055179 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.055140    2583 scope.go:117] "RemoveContainer" containerID="6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839"
Jul 06 23:58:46.055366 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:46.055351    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839\": container with ID starting with 6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839 not found: ID does not exist" containerID="6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839"
Jul 06 23:58:46.055411 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.055371    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839"} err="failed to get container status \"6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839\": rpc error: code = NotFound desc = could not find container \"6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839\": container with ID starting with 6dfa69e44bba11e19a428af4d618aaa041fb371671718ec6202b95e136fc9839 not found: ID does not exist"
Jul 06 23:58:46.088972 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.088945    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:46.089278 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089263    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="prom-label-proxy"
Jul 06 23:58:46.089351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089281    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="prom-label-proxy"
Jul 06 23:58:46.089351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089296    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="init-config-reloader"
Jul 06 23:58:46.089351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089304    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="init-config-reloader"
Jul 06 23:58:46.089351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089329    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-web"
Jul 06 23:58:46.089351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089339    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-web"
Jul 06 23:58:46.089351 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089350    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089358    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089371    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-metric"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089380    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-metric"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089390    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="alertmanager"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089400    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="alertmanager"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089411    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="config-reloader"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089420    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="config-reloader"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089491    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="config-reloader"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089503    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="prom-label-proxy"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089514    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089542    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-web"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089554    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="alertmanager"
Jul 06 23:58:46.089673 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.089565    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" containerName="kube-rbac-proxy-metric"
Jul 06 23:58:46.093336 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.093319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.096115 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 23:58:46.096181 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096100    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 23:58:46.096643 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096568    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 23:58:46.096643 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096594    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 23:58:46.096643 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096639    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 23:58:46.096821 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096595    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 23:58:46.096821 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096595    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 23:58:46.096821 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096640    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 23:58:46.096821 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.096594    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8bxds\""
Jul 06 23:58:46.101686 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.101668    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 23:58:46.113150 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.113123    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:46.156209 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.156182    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb3680e4-50ef-47b7-8743-aa8dd81a8585" path="/var/lib/kubelet/pods/fb3680e4-50ef-47b7-8743-aa8dd81a8585/volumes"
Jul 06 23:58:46.233872 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.233838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.233872 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.233875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.233892    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vsl5\" (UniqueName: \"kubernetes.io/projected/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-kube-api-access-8vsl5\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.233914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.233982    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234011    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234082 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234055    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-web-config\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234312 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234312 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234112    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234312 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234148    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-config-out\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234312 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-config-volume\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.234312 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.234201    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335383 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335383 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335383 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-web-config\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-config-out\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-config-volume\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335650 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.335973 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.335704    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.336999 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.336968    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.338373 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.338336    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.338483 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.338402    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vsl5\" (UniqueName: \"kubernetes.io/projected/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-kube-api-access-8vsl5\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.338904 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.338874    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.339349 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.339316    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-web-config\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.339427 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.339406    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-config-volume\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.339508 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.339483    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.339809 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.339743    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.339929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.339841    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.343882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.340154    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.343882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.340693    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.343882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.340846    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-config-out\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.343882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.341690    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.343882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.341844    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.348100 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.348076    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vsl5\" (UniqueName: \"kubernetes.io/projected/fcf97620-1bc7-4916-ae42-f79d6f3d36c6-kube-api-access-8vsl5\") pod \"alertmanager-main-0\" (UID: \"fcf97620-1bc7-4916-ae42-f79d6f3d36c6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.402721 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.402689    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:46.545213 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:46.545189    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:46.547519 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:58:46.547485    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf97620_1bc7_4916_ae42_f79d6f3d36c6.slice/crio-4e1393986116bf8fe6d7a104be8bcc2390b98127f9b24ce10e64b6bb629a87d8 WatchSource:0}: Error finding container 4e1393986116bf8fe6d7a104be8bcc2390b98127f9b24ce10e64b6bb629a87d8: Status 404 returned error can't find the container with id 4e1393986116bf8fe6d7a104be8bcc2390b98127f9b24ce10e64b6bb629a87d8
Jul 06 23:58:47.008170 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:47.008138    2583 generic.go:356] "Generic (PLEG): container finished" podID="fcf97620-1bc7-4916-ae42-f79d6f3d36c6" containerID="9d45816f65d58b71c24a0c235a9f5778362a1feeee87e0c7c959a509cfc2e7bc" exitCode=0
Jul 06 23:58:47.008306 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:47.008177    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerDied","Data":"9d45816f65d58b71c24a0c235a9f5778362a1feeee87e0c7c959a509cfc2e7bc"}
Jul 06 23:58:47.008306 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:47.008227    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"4e1393986116bf8fe6d7a104be8bcc2390b98127f9b24ce10e64b6bb629a87d8"}
Jul 06 23:58:48.015564 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.015511    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"cf929c49fb7ad22e692d50ea85fa207f5d7971448ffb8be15cc93290a5b2d805"}
Jul 06 23:58:48.015929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.015594    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"2c13b9e7c11bde243f1baf07f18a6f2ccf0076b480455b1f7a6934d1f5c2e08e"}
Jul 06 23:58:48.015929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.015620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"9fb5974ad50598cf0f9237f69e4fa2f9fb4e3c7de3c47410d4311c732bbfc94a"}
Jul 06 23:58:48.015929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.015644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"fdff96cc920bdfb3021f8a543e7fee3775ef066e566061b686850e67ea55cf0a"}
Jul 06 23:58:48.015929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.015666    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"a6117dcb30176025e516bc9f7b40ba874ec393ad79f8accd38469f83d1fb0fd3"}
Jul 06 23:58:48.015929 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.015689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fcf97620-1bc7-4916-ae42-f79d6f3d36c6","Type":"ContainerStarted","Data":"b76145444e959a58efc46c80a9c1090c948a0d32bd58f6e330e1c9e1aa2c6861"}
Jul 06 23:58:48.046952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.046897    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.0257837 podStartE2EDuration="2.046878274s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.094574ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:48.045282399 +0000 UTC m=+152.410586144" watchObservedRunningTime="2026-07-06 23:58:48.046878274 +0000 UTC m=+152.412182020"
Jul 06 23:58:48.489194 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489160    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:58:48.489627 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489595    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="prometheus" containerID="cri-o://2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2" gracePeriod=600
Jul 06 23:58:48.489727 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489618    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="thanos-sidecar" containerID="cri-o://57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc" gracePeriod=600
Jul 06 23:58:48.489727 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489649    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="config-reloader" containerID="cri-o://ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab" gracePeriod=600
Jul 06 23:58:48.489828 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489604    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-web" containerID="cri-o://59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf" gracePeriod=600
Jul 06 23:58:48.489828 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489610    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-thanos" containerID="cri-o://374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a" gracePeriod=600
Jul 06 23:58:48.489828 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.489776    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy" containerID="cri-o://1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5" gracePeriod=600
Jul 06 23:58:48.742795 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.742418    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:48.864936 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.864905    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v45bd\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-kube-api-access-v45bd\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.864944    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-2\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.864965    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-config-out\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.864990    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-config\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865010    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-web-config\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865032    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-0\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865090    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-serving-certs-ca-bundle\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865119 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865114    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-db\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865147    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-kube-rbac-proxy\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865177    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-grpc-tls\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865214    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-tls\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865239    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-tls-assets\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865264    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-metrics-client-certs\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865325    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-kubelet-serving-ca-bundle\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865363    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865399    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-thanos-prometheus-http-client-file\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-1\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865457    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-trusted-ca-bundle\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.865493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865493    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-metrics-client-ca\") pod \"937977c8-f6de-47f3-a6f9-885f4f5083a1\" (UID: \"937977c8-f6de-47f3-a6f9-885f4f5083a1\") "
Jul 06 23:58:48.867307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.865313    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.866091    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.866272    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.866634    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.867033    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867307 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.867273    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867691 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.867506    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:48.867892 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.867799    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:48.867892 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.867834    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-config" (OuterVolumeSpecName: "config") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.869505 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.869395    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.869505 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.869468    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-config-out" (OuterVolumeSpecName: "config-out") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:48.869880 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.869844    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.870268 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.870230    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.870397 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.870370    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.870697 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.870664    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.870788 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.870769    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.870930 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.870910    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-kube-api-access-v45bd" (OuterVolumeSpecName: "kube-api-access-v45bd") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "kube-api-access-v45bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:48.871003 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.870979    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.871137 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.871119    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:48.879506 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.879488    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-web-config" (OuterVolumeSpecName: "web-config") pod "937977c8-f6de-47f3-a6f9-885f4f5083a1" (UID: "937977c8-f6de-47f3-a6f9-885f4f5083a1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:48.966678 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966643    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966678 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966672    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-db\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966678 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966684    2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-kube-rbac-proxy\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966694    2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-grpc-tls\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966703    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966712    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-tls-assets\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966720    2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-metrics-client-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966728    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966737    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966746    2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966753    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966761    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966769    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-configmap-metrics-client-ca\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966777    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v45bd\" (UniqueName: \"kubernetes.io/projected/937977c8-f6de-47f3-a6f9-885f4f5083a1-kube-api-access-v45bd\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966784    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966792    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/937977c8-f6de-47f3-a6f9-885f4f5083a1-config-out\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966800    2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-config\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966807    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-web-config\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966816    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/937977c8-f6de-47f3-a6f9-885f4f5083a1-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:48.966888 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:48.966823    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/937977c8-f6de-47f3-a6f9-885f4f5083a1-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022716    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a" exitCode=0
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022743    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5" exitCode=0
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022750    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf" exitCode=0
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022755    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc" exitCode=0
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022761    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab" exitCode=0
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022767    2583 generic.go:356] "Generic (PLEG): container finished" podID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2" exitCode=0
Jul 06 23:58:49.022806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022795    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"}
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022878    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"}
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022803    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022920    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"}
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"}
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022972    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.022989    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"}
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.023015    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"}
Jul 06 23:58:49.023493 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.023044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"937977c8-f6de-47f3-a6f9-885f4f5083a1","Type":"ContainerDied","Data":"f6976a924bd77b0813d1628965a8898c001282c71c514e49c5a9f552d8d51fcb"}
Jul 06 23:58:49.031556 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.031519    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.038536 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.038507    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.045170 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.045155    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.049677 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.049655    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:58:49.052922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.052905    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.057342 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.057327    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:58:49.060048 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.060030    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.066882 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.066867    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.076642 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.076620    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.076901 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.076883    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.076949 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.076907    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"} err="failed to get container status \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist"
Jul 06 23:58:49.076949 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.076923    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.077172 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.077155    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.077235 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077182    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"} err="failed to get container status \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist"
Jul 06 23:58:49.077235 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077205    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.077408 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.077392    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.077450 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077410    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"} err="failed to get container status \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist"
Jul 06 23:58:49.077450 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077423    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.077684 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.077665    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.077761 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077687    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"} err="failed to get container status \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist"
Jul 06 23:58:49.077761 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077700    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.077950 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.077933    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.077983 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077959    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"} err="failed to get container status \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist"
Jul 06 23:58:49.077983 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.077975    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.078192 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.078177    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.078241 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.078195    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"} err="failed to get container status \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist"
Jul 06 23:58:49.078241 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.078209    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.078462 ip-10-0-141-68 kubenswrapper[2583]: E0706 23:58:49.078443    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.078552 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.078469    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"} err="failed to get container status \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist"
Jul 06 23:58:49.078552 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.078490    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.078880 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.078857    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"} err="failed to get container status \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist"
Jul 06 23:58:49.078926 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.078881    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.079188 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079171    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"} err="failed to get container status \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist"
Jul 06 23:58:49.079188 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079189    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.079549 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079508    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"} err="failed to get container status \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist"
Jul 06 23:58:49.079603 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079546    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.079789 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079771    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"} err="failed to get container status \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist"
Jul 06 23:58:49.079833 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079789    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.079978 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079954    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"} err="failed to get container status \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist"
Jul 06 23:58:49.079978 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.079976    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.080170 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080155    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"} err="failed to get container status \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist"
Jul 06 23:58:49.080215 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080171    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.080382 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080367    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"} err="failed to get container status \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist"
Jul 06 23:58:49.080425 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080382    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.080625 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080606    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"} err="failed to get container status \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist"
Jul 06 23:58:49.080665 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080626    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.080848 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080826    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"} err="failed to get container status \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist"
Jul 06 23:58:49.080848 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.080843    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.081029 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081014    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"} err="failed to get container status \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist"
Jul 06 23:58:49.081067 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081029    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.081259 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081245    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"} err="failed to get container status \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist"
Jul 06 23:58:49.081299 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081261    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.081573 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081465    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"} err="failed to get container status \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist"
Jul 06 23:58:49.081573 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081503    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.081915 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081890    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"} err="failed to get container status \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist"
Jul 06 23:58:49.081915 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.081916    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.082426 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.082401    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"} err="failed to get container status \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist"
Jul 06 23:58:49.082544 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.082428    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.082788 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.082749    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"} err="failed to get container status \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist"
Jul 06 23:58:49.082788 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.082769    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.083139 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083111    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"} err="failed to get container status \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist"
Jul 06 23:58:49.083139 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083139    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.083377 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083346    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"} err="failed to get container status \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist"
Jul 06 23:58:49.083377 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083374    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.083613 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083596    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"} err="failed to get container status \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist"
Jul 06 23:58:49.083666 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083630    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.083714 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083675    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:58:49.083857 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083838    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"} err="failed to get container status \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist"
Jul 06 23:58:49.083904 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083859    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.083985 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083971    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="thanos-sidecar"
Jul 06 23:58:49.084032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.083989    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="thanos-sidecar"
Jul 06 23:58:49.084032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084002    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy"
Jul 06 23:58:49.084032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084011    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy"
Jul 06 23:58:49.084032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084024    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="prometheus"
Jul 06 23:58:49.084032 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084029    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="prometheus"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084036    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="init-config-reloader"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084041    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="init-config-reloader"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084047    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"} err="failed to get container status \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084072    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084056    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-web"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084110    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-web"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084123    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-thanos"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084129    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-thanos"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084138    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="config-reloader"
Jul 06 23:58:49.084206 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084144    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="config-reloader"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084220    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084234    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-web"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084244    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="thanos-sidecar"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084252    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="prometheus"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084260    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="config-reloader"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084268    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" containerName="kube-rbac-proxy-thanos"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084275    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"} err="failed to get container status \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084292    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084480    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"} err="failed to get container status \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist"
Jul 06 23:58:49.084585 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084503    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.084940 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084741    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"} err="failed to get container status \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist"
Jul 06 23:58:49.084940 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.084758    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.085043 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085026    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"} err="failed to get container status \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist"
Jul 06 23:58:49.085079 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085044    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.085250 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085234    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"} err="failed to get container status \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist"
Jul 06 23:58:49.085289 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085250    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.085491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085463    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"} err="failed to get container status \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist"
Jul 06 23:58:49.085491 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085483    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.085691 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085669    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"} err="failed to get container status \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist"
Jul 06 23:58:49.085753 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085693    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.085983 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085955    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"} err="failed to get container status \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist"
Jul 06 23:58:49.085983 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.085984    2583 scope.go:117] "RemoveContainer" containerID="374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"
Jul 06 23:58:49.086356 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.086242    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a"} err="failed to get container status \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": rpc error: code = NotFound desc = could not find container \"374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a\": container with ID starting with 374f8ae3f5c5ae6bd3d7187f372afb4ff1ae098201db330940d41387b772c89a not found: ID does not exist"
Jul 06 23:58:49.086356 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.086273    2583 scope.go:117] "RemoveContainer" containerID="1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"
Jul 06 23:58:49.086566 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.086542    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5"} err="failed to get container status \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": rpc error: code = NotFound desc = could not find container \"1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5\": container with ID starting with 1f851ac6e51e1a6289bda9ff053b8468e70074b67b25c443a838e951133f69e5 not found: ID does not exist"
Jul 06 23:58:49.086643 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.086568    2583 scope.go:117] "RemoveContainer" containerID="59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"
Jul 06 23:58:49.087080 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087058    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf"} err="failed to get container status \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": rpc error: code = NotFound desc = could not find container \"59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf\": container with ID starting with 59faf8325a681640b437aab491ec540bc351f055be193b6375a6da8ea7cda2bf not found: ID does not exist"
Jul 06 23:58:49.087080 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087077    2583 scope.go:117] "RemoveContainer" containerID="57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"
Jul 06 23:58:49.087319 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087302    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc"} err="failed to get container status \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": rpc error: code = NotFound desc = could not find container \"57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc\": container with ID starting with 57b9f6d1f6a2b44731fb4cc17ccdecc732a8594bc3c9547b35b73224876567bc not found: ID does not exist"
Jul 06 23:58:49.087319 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087318    2583 scope.go:117] "RemoveContainer" containerID="ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"
Jul 06 23:58:49.087565 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087547    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab"} err="failed to get container status \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": rpc error: code = NotFound desc = could not find container \"ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab\": container with ID starting with ab4fef2c3e076cd2eabdf261725a04e3719ae5a281cbb4486c97579c43905bab not found: ID does not exist"
Jul 06 23:58:49.087613 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087565    2583 scope.go:117] "RemoveContainer" containerID="2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"
Jul 06 23:58:49.087763 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087747    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2"} err="failed to get container status \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": rpc error: code = NotFound desc = could not find container \"2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2\": container with ID starting with 2ece7e55094e0f020e1d900b71ba8e7e32e0f681483fee10de090b5bc0f4eae2 not found: ID does not exist"
Jul 06 23:58:49.087814 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087764    2583 scope.go:117] "RemoveContainer" containerID="6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"
Jul 06 23:58:49.087963 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.087946    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac"} err="failed to get container status \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": rpc error: code = NotFound desc = could not find container \"6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac\": container with ID starting with 6a47069dbb6d25319009ffaa8ea2be983e12664d37c1fbc0c483d3bf3e95dcac not found: ID does not exist"
Jul 06 23:58:49.089392 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.089377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.092073 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.092055    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 23:58:49.092165 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.092110    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 23:58:49.092396 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.092381    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 23:58:49.092396 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.092392    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 23:58:49.092516 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.092407    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 23:58:49.092516 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.092387    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 23:58:49.093125 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.093106    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 23:58:49.093258 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.093136    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 23:58:49.093258 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.093166    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 23:58:49.093258 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.093245    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9zxpb\""
Jul 06 23:58:49.093484 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.093462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 23:58:49.093806 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.093792    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 23:58:49.094430 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.094415    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 23:58:49.094544 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.094458    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 23:58:49.094671 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.094658    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b4su5h3stafmr\""
Jul 06 23:58:49.095952 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.095913    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 23:58:49.097773 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.097755    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 23:58:49.103327 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.103309    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:58:49.167693 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167662    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167693 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167690    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs8nk\" (UniqueName: \"kubernetes.io/projected/7a11914c-6465-455b-98cd-34522022977e-kube-api-access-fs8nk\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a11914c-6465-455b-98cd-34522022977e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.167922 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168163 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.167978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168163 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168272 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168272 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168262    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168369 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168331    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168419 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168393    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168516 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168516 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-config\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168641 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168575    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a11914c-6465-455b-98cd-34522022977e-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168641 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168595    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.168641 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.168614    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.269950 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.269910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a11914c-6465-455b-98cd-34522022977e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270104 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.269955    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270104 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.269981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270197 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270158    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270258 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270258 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270360 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270268    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270360 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270360 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270331    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270360 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270393    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-config\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a11914c-6465-455b-98cd-34522022977e-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270586 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270562    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270991 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270607    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270991 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270991 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270991 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs8nk\" (UniqueName: \"kubernetes.io/projected/7a11914c-6465-455b-98cd-34522022977e-kube-api-access-fs8nk\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270991 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.270991 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.270935    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.271288 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.271178    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.271288 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.271183    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.272192 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.271507    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.272192 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.271855    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.272192 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.272139    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.272676 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.272653    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.273225 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.273163    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a11914c-6465-455b-98cd-34522022977e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.273316 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.273205    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.273395 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.273278    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.274168 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.273687    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.274168 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.273935    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.275467 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.275401    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a11914c-6465-455b-98cd-34522022977e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.275467 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.275426    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.275631 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.275513    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-config\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.275699 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.275679    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.275814 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.275800    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a11914c-6465-455b-98cd-34522022977e-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.275932 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.275917    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.276313 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.276292    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a11914c-6465-455b-98cd-34522022977e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.282612 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.282596    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs8nk\" (UniqueName: \"kubernetes.io/projected/7a11914c-6465-455b-98cd-34522022977e-kube-api-access-fs8nk\") pod \"prometheus-k8s-0\" (UID: \"7a11914c-6465-455b-98cd-34522022977e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.399429 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.399392    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:49.539648 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:49.539622    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:58:49.541755 ip-10-0-141-68 kubenswrapper[2583]: W0706 23:58:49.541725    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a11914c_6465_455b_98cd_34522022977e.slice/crio-e56cb677819620f8dfc2109ab3e84013a5cacfe555b859d8a5253f4754be5bb8 WatchSource:0}: Error finding container e56cb677819620f8dfc2109ab3e84013a5cacfe555b859d8a5253f4754be5bb8: Status 404 returned error can't find the container with id e56cb677819620f8dfc2109ab3e84013a5cacfe555b859d8a5253f4754be5bb8
Jul 06 23:58:50.030364 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:50.030326    2583 generic.go:356] "Generic (PLEG): container finished" podID="7a11914c-6465-455b-98cd-34522022977e" containerID="6eca54d8fd5ba3f9334c92ea935c4a57eba52aee2a578456855673f3b559899e" exitCode=0
Jul 06 23:58:50.030749 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:50.030384    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerDied","Data":"6eca54d8fd5ba3f9334c92ea935c4a57eba52aee2a578456855673f3b559899e"}
Jul 06 23:58:50.030749 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:50.030465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"e56cb677819620f8dfc2109ab3e84013a5cacfe555b859d8a5253f4754be5bb8"}
Jul 06 23:58:50.162511 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:50.162479    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937977c8-f6de-47f3-a6f9-885f4f5083a1" path="/var/lib/kubelet/pods/937977c8-f6de-47f3-a6f9-885f4f5083a1/volumes"
Jul 06 23:58:51.039300 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.039262    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"5cfb6c6d1a8c0fdc52b2084746e2167eeffe1ee46b2ffccb19412b89249902f3"}
Jul 06 23:58:51.039813 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.039340    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"4b98949ce4cde5e75c9b89f86b55a2f337c0ca8b5c3a1a106a6a9c35f56a6df8"}
Jul 06 23:58:51.039813 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.039379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"75f5d3f0aea527df0ee720e99fa0032b4cafff52a50d1c72bf7052d5807a82e6"}
Jul 06 23:58:51.039813 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.039422    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"8328f9390575c54d0786b601908aeab2a01e94a2944c4c5f3f911c3392849767"}
Jul 06 23:58:51.039813 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.039464    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"fefe0e7b712f621c14c16661b16e975f2f22cffac8036fa0d2e3ebed439047e4"}
Jul 06 23:58:51.039813 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.039502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a11914c-6465-455b-98cd-34522022977e","Type":"ContainerStarted","Data":"26cb68bab0666fe817ea3d68a8828b93905dd389c1b6acbbf1a7b76661fe6685"}
Jul 06 23:58:51.071873 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:51.071803    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.048815775 podStartE2EDuration="2.071781027s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.965252ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:51.069453478 +0000 UTC m=+155.434757224" watchObservedRunningTime="2026-07-06 23:58:51.071781027 +0000 UTC m=+155.437084776"
Jul 06 23:58:54.399804 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:58:54.399750    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:49.400362 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:59:49.400328    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:49.416354 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:59:49.416329    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:50.282071 ip-10-0-141-68 kubenswrapper[2583]: I0706 23:59:50.282044    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 00:00:54.529120 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.529034    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"]
Jul 07 00:00:54.531991 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.531967    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.535583 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.535559    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 00:00:54.538340 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.538317    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 00:00:54.539984 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.539964    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-m2qsk\""
Jul 07 00:00:54.540254 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.540239    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 00:00:54.540389 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.540369    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 00:00:54.540618 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.540605    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 00:00:54.551068 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.551036    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvx8t\" (UniqueName: \"kubernetes.io/projected/f0feca0b-94dc-4c1b-add4-20f18f358220-kube-api-access-nvx8t\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.551203 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.551089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f0feca0b-94dc-4c1b-add4-20f18f358220-manager-config\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.551203 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.551192    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0feca0b-94dc-4c1b-add4-20f18f358220-cert\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.551314 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.551223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0feca0b-94dc-4c1b-add4-20f18f358220-metrics-cert\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.551667 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.551646    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"]
Jul 07 00:00:54.652212 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.652170    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0feca0b-94dc-4c1b-add4-20f18f358220-cert\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.652212 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.652215    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0feca0b-94dc-4c1b-add4-20f18f358220-metrics-cert\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.652474 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.652246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvx8t\" (UniqueName: \"kubernetes.io/projected/f0feca0b-94dc-4c1b-add4-20f18f358220-kube-api-access-nvx8t\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.652474 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.652404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f0feca0b-94dc-4c1b-add4-20f18f358220-manager-config\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.653122 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.653074    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f0feca0b-94dc-4c1b-add4-20f18f358220-manager-config\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.654916 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.654895    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0feca0b-94dc-4c1b-add4-20f18f358220-metrics-cert\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.655016 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.654904    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0feca0b-94dc-4c1b-add4-20f18f358220-cert\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.667986 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.667956    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvx8t\" (UniqueName: \"kubernetes.io/projected/f0feca0b-94dc-4c1b-add4-20f18f358220-kube-api-access-nvx8t\") pod \"lws-controller-manager-59b45dbd94-bvht7\" (UID: \"f0feca0b-94dc-4c1b-add4-20f18f358220\") " pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.842417 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.842378    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:54.997245 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:54.997212    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"]
Jul 07 00:00:55.000251 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:00:55.000215    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0feca0b_94dc_4c1b_add4_20f18f358220.slice/crio-2e98be67c93745a9bf6696958b407a04af9c1e7d3abcad2f3971f33e1c0779d7 WatchSource:0}: Error finding container 2e98be67c93745a9bf6696958b407a04af9c1e7d3abcad2f3971f33e1c0779d7: Status 404 returned error can't find the container with id 2e98be67c93745a9bf6696958b407a04af9c1e7d3abcad2f3971f33e1c0779d7
Jul 07 00:00:55.520458 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:55.520426    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7" event={"ID":"f0feca0b-94dc-4c1b-add4-20f18f358220","Type":"ContainerStarted","Data":"2e98be67c93745a9bf6696958b407a04af9c1e7d3abcad2f3971f33e1c0779d7"}
Jul 07 00:00:56.988592 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.988558    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"]
Jul 07 00:00:56.994551 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.994501    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:56.998595 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.998418    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p27dv\""
Jul 07 00:00:56.998768 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.998731    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 00:00:56.999238 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.999210    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 00:00:56.999737 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.999661    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 00:00:56.999737 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:56.999736    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 00:00:57.025560 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.025504    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"]
Jul 07 00:00:57.070644 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.070608    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9cbfe34d-8232-4c98-82ae-f09c99b79056-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.070799 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.070659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9cbfe34d-8232-4c98-82ae-f09c99b79056-manifests\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.070799 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.070707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cbfe34d-8232-4c98-82ae-f09c99b79056-tmp\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.070906 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.070809    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdms\" (UniqueName: \"kubernetes.io/projected/9cbfe34d-8232-4c98-82ae-f09c99b79056-kube-api-access-jfdms\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.070906 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.070846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9cbfe34d-8232-4c98-82ae-f09c99b79056-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.171898 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.171860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9cbfe34d-8232-4c98-82ae-f09c99b79056-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.171898 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.171899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9cbfe34d-8232-4c98-82ae-f09c99b79056-manifests\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.172146 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.171953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cbfe34d-8232-4c98-82ae-f09c99b79056-tmp\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.172146 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.172017    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdms\" (UniqueName: \"kubernetes.io/projected/9cbfe34d-8232-4c98-82ae-f09c99b79056-kube-api-access-jfdms\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.172146 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.172051    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9cbfe34d-8232-4c98-82ae-f09c99b79056-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.172427 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.172398    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9cbfe34d-8232-4c98-82ae-f09c99b79056-manifests\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.174740 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.174702    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cbfe34d-8232-4c98-82ae-f09c99b79056-tmp\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.175080 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.175057    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9cbfe34d-8232-4c98-82ae-f09c99b79056-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.175167 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.175090    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9cbfe34d-8232-4c98-82ae-f09c99b79056-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.190208 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.190186    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdms\" (UniqueName: \"kubernetes.io/projected/9cbfe34d-8232-4c98-82ae-f09c99b79056-kube-api-access-jfdms\") pod \"opendatahub-operator-controller-manager-5d7997cf4b-sdvxl\" (UID: \"9cbfe34d-8232-4c98-82ae-f09c99b79056\") " pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.308629 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.308554    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:00:57.999184 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:57.999150    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"]
Jul 07 00:00:58.002910 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:00:58.002880    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cbfe34d_8232_4c98_82ae_f09c99b79056.slice/crio-05eb73f37809d796c60aa779bdb627419202d4a7c67e69af9451d5c3cb201ea3 WatchSource:0}: Error finding container 05eb73f37809d796c60aa779bdb627419202d4a7c67e69af9451d5c3cb201ea3: Status 404 returned error can't find the container with id 05eb73f37809d796c60aa779bdb627419202d4a7c67e69af9451d5c3cb201ea3
Jul 07 00:00:58.535310 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:58.535271    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl" event={"ID":"9cbfe34d-8232-4c98-82ae-f09c99b79056","Type":"ContainerStarted","Data":"05eb73f37809d796c60aa779bdb627419202d4a7c67e69af9451d5c3cb201ea3"}
Jul 07 00:00:58.536687 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:58.536662    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7" event={"ID":"f0feca0b-94dc-4c1b-add4-20f18f358220","Type":"ContainerStarted","Data":"f4180a6c2cc595d5176e9d6098874a1665951c69ce2892a5b0bfd88d65055fe1"}
Jul 07 00:00:58.536849 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:58.536769    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:00:58.565914 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:00:58.565865    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7" podStartSLOduration=1.695463988 podStartE2EDuration="4.56584885s" totalImagesPullingTime="2.870384862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:58.564522001 +0000 UTC m=+282.929825747" watchObservedRunningTime="2026-07-07 00:00:58.56584885 +0000 UTC m=+282.931152595"
Jul 07 00:01:02.560321 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:02.560281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl" event={"ID":"9cbfe34d-8232-4c98-82ae-f09c99b79056","Type":"ContainerStarted","Data":"1794d33901190ca39f0d437137fa805e24139fb03325fb2b383c84099c5e111b"}
Jul 07 00:01:03.566591 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:03.566553    2583 generic.go:356] "Generic (PLEG): container finished" podID="9cbfe34d-8232-4c98-82ae-f09c99b79056" containerID="1794d33901190ca39f0d437137fa805e24139fb03325fb2b383c84099c5e111b" exitCode=0
Jul 07 00:01:03.567065 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:03.566618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl" event={"ID":"9cbfe34d-8232-4c98-82ae-f09c99b79056","Type":"ContainerDied","Data":"1794d33901190ca39f0d437137fa805e24139fb03325fb2b383c84099c5e111b"}
Jul 07 00:01:04.572538 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:04.572504    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl" event={"ID":"9cbfe34d-8232-4c98-82ae-f09c99b79056","Type":"ContainerStarted","Data":"06063e73b0481e9a7024819f3bcd8d19dba899bda51b5d18368bd62e932c625f"}
Jul 07 00:01:04.572931 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:04.572624    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:01:04.600977 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:04.600923    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl" podStartSLOduration=3.363046862 podStartE2EDuration="8.60090409s" totalImagesPullingTime="4.155746857s" totalInitContainerRuntime="1.082110371s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:04.599325128 +0000 UTC m=+288.964628922" watchObservedRunningTime="2026-07-07 00:01:04.60090409 +0000 UTC m=+288.966207836"
Jul 07 00:01:09.546328 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:09.546290    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59b45dbd94-bvht7"
Jul 07 00:01:15.579801 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:15.579766    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d7997cf4b-sdvxl"
Jul 07 00:01:16.031544 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:16.031501    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:01:16.036465 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:16.036438    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:01:16.037610 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:16.037591    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 00:01:17.924480 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.924451    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"]
Jul 07 00:01:17.927611 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.927591    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:17.931131 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.931109    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 00:01:17.931293 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.931171    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 00:01:17.931390 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.931376    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 00:01:17.931469 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.931376    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4x894\""
Jul 07 00:01:17.931734 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.931713    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 00:01:17.943857 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:17.943824    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"]
Jul 07 00:01:18.055920 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.055885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f74e76-027a-41d9-a240-6e7248af829d-tls-certs\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.056082 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.055959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rql\" (UniqueName: \"kubernetes.io/projected/c6f74e76-027a-41d9-a240-6e7248af829d-kube-api-access-94rql\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.056082 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.055980    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f74e76-027a-41d9-a240-6e7248af829d-tmp\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.156337 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.156298    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rql\" (UniqueName: \"kubernetes.io/projected/c6f74e76-027a-41d9-a240-6e7248af829d-kube-api-access-94rql\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.156337 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.156329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f74e76-027a-41d9-a240-6e7248af829d-tmp\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.156620 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.156367    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f74e76-027a-41d9-a240-6e7248af829d-tls-certs\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.158878 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.158844    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f74e76-027a-41d9-a240-6e7248af829d-tmp\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.159078 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.159059    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6f74e76-027a-41d9-a240-6e7248af829d-tls-certs\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.170833 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.170811    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rql\" (UniqueName: \"kubernetes.io/projected/c6f74e76-027a-41d9-a240-6e7248af829d-kube-api-access-94rql\") pod \"kube-auth-proxy-7697f65cb7-fd4qc\" (UID: \"c6f74e76-027a-41d9-a240-6e7248af829d\") " pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.238344 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.238321    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"
Jul 07 00:01:18.400376 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.400347    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc"]
Jul 07 00:01:18.402182 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:01:18.402155    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f74e76_027a_41d9_a240_6e7248af829d.slice/crio-29bb5e1df61a767fddbabdf7a7905fabb244d87cf1a09ed59cfe30021d311249 WatchSource:0}: Error finding container 29bb5e1df61a767fddbabdf7a7905fabb244d87cf1a09ed59cfe30021d311249: Status 404 returned error can't find the container with id 29bb5e1df61a767fddbabdf7a7905fabb244d87cf1a09ed59cfe30021d311249
Jul 07 00:01:18.404007 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.403988    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:01:18.632459 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:18.632378    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc" event={"ID":"c6f74e76-027a-41d9-a240-6e7248af829d","Type":"ContainerStarted","Data":"29bb5e1df61a767fddbabdf7a7905fabb244d87cf1a09ed59cfe30021d311249"}
Jul 07 00:01:22.652183 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:22.652145    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc" event={"ID":"c6f74e76-027a-41d9-a240-6e7248af829d","Type":"ContainerStarted","Data":"6ca83852bd4e2a9375086db1d8c38edcf080bfb84e2eae196d2bf82cf7c234cc"}
Jul 07 00:01:22.688219 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:01:22.688161    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7697f65cb7-fd4qc" podStartSLOduration=2.144880848 podStartE2EDuration="5.688138348s" totalImagesPullingTime="3.5432575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:22.68723817 +0000 UTC m=+307.052541915" watchObservedRunningTime="2026-07-07 00:01:22.688138348 +0000 UTC m=+307.053442095"
Jul 07 00:03:02.353393 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.353353    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"]
Jul 07 00:03:02.356772 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.356750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.362974 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.362946    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cjtgc\""
Jul 07 00:03:02.363675 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.363345    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 00:03:02.364057 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.364038    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 00:03:02.364362 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.364346    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 00:03:02.368489 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.368470    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 00:03:02.372015 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.371990    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"]
Jul 07 00:03:02.484072 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.484031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7bf67cb-f796-47dc-ba58-18f1d163786b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.484240 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.484116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqww5\" (UniqueName: \"kubernetes.io/projected/e7bf67cb-f796-47dc-ba58-18f1d163786b-kube-api-access-dqww5\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.484240 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.484161    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7bf67cb-f796-47dc-ba58-18f1d163786b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.584997 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.584941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7bf67cb-f796-47dc-ba58-18f1d163786b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.585187 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.585050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqww5\" (UniqueName: \"kubernetes.io/projected/e7bf67cb-f796-47dc-ba58-18f1d163786b-kube-api-access-dqww5\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.585187 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.585104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7bf67cb-f796-47dc-ba58-18f1d163786b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.585812 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.585792    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7bf67cb-f796-47dc-ba58-18f1d163786b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.587614 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.587589    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7bf67cb-f796-47dc-ba58-18f1d163786b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.595320 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.595296    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqww5\" (UniqueName: \"kubernetes.io/projected/e7bf67cb-f796-47dc-ba58-18f1d163786b-kube-api-access-dqww5\") pod \"kuadrant-console-plugin-66b5774985-tfkrq\" (UID: \"e7bf67cb-f796-47dc-ba58-18f1d163786b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.667091 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.667008    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"
Jul 07 00:03:02.807597 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:02.807424    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq"]
Jul 07 00:03:02.809808 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:03:02.809783    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7bf67cb_f796_47dc_ba58_18f1d163786b.slice/crio-f4851bed99ce026eacfd25d32cca8bfdf82c35fdd959ba2374517cb170322313 WatchSource:0}: Error finding container f4851bed99ce026eacfd25d32cca8bfdf82c35fdd959ba2374517cb170322313: Status 404 returned error can't find the container with id f4851bed99ce026eacfd25d32cca8bfdf82c35fdd959ba2374517cb170322313
Jul 07 00:03:03.109088 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:03.109054    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq" event={"ID":"e7bf67cb-f796-47dc-ba58-18f1d163786b","Type":"ContainerStarted","Data":"f4851bed99ce026eacfd25d32cca8bfdf82c35fdd959ba2374517cb170322313"}
Jul 07 00:03:25.225881 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:25.225847    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq" event={"ID":"e7bf67cb-f796-47dc-ba58-18f1d163786b","Type":"ContainerStarted","Data":"7d7b6a738f99ea328b99b50a4292ba4ec3132bfefcee946ee59d9aeafb1d88be"}
Jul 07 00:03:25.266868 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:25.266675    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfkrq" podStartSLOduration=0.967377351 podStartE2EDuration="23.266658487s" totalImagesPullingTime="22.299281136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:25.266584617 +0000 UTC m=+429.631888375" watchObservedRunningTime="2026-07-07 00:03:25.266658487 +0000 UTC m=+429.631962234"
Jul 07 00:03:48.363402 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.363295    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:03:48.553785 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.553744    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:03:48.553785 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.553742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.554032 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.553811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:03:48.556439 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.556419    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 00:03:48.701755 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.701683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rmt\" (UniqueName: \"kubernetes.io/projected/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-kube-api-access-l4rmt\") pod \"limitador-limitador-64c94fd64f-qrprl\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.701755 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.701742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-config-file\") pod \"limitador-limitador-64c94fd64f-qrprl\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.802681 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.802642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rmt\" (UniqueName: \"kubernetes.io/projected/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-kube-api-access-l4rmt\") pod \"limitador-limitador-64c94fd64f-qrprl\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.802854 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.802716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-config-file\") pod \"limitador-limitador-64c94fd64f-qrprl\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.803302 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.803282    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-config-file\") pod \"limitador-limitador-64c94fd64f-qrprl\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.814428 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.814398    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rmt\" (UniqueName: \"kubernetes.io/projected/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-kube-api-access-l4rmt\") pod \"limitador-limitador-64c94fd64f-qrprl\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:48.863415 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:48.863368    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:49.021080 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.021048    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:03:49.024172 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:03:49.024146    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e952cae_6586_4e1a_8ef2_e1837eb33fa9.slice/crio-8c17dedf6faafabdbd66ab915c8ef3b7fcf11eab01f25637ef840ff059023990 WatchSource:0}: Error finding container 8c17dedf6faafabdbd66ab915c8ef3b7fcf11eab01f25637ef840ff059023990: Status 404 returned error can't find the container with id 8c17dedf6faafabdbd66ab915c8ef3b7fcf11eab01f25637ef840ff059023990
Jul 07 00:03:49.197015 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.196980    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-x49x5"]
Jul 07 00:03:49.240315 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.240291    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-x49x5"]
Jul 07 00:03:49.240466 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.240387    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:49.243721 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.243698    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f6mp6\""
Jul 07 00:03:49.327324 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.327286    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl" event={"ID":"0e952cae-6586-4e1a-8ef2-e1837eb33fa9","Type":"ContainerStarted","Data":"8c17dedf6faafabdbd66ab915c8ef3b7fcf11eab01f25637ef840ff059023990"}
Jul 07 00:03:49.408429 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.408393    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tthqn\" (UniqueName: \"kubernetes.io/projected/fa654b83-ef38-416c-a8ee-a65d4bede873-kube-api-access-tthqn\") pod \"authorino-58cd955675-x49x5\" (UID: \"fa654b83-ef38-416c-a8ee-a65d4bede873\") " pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:49.509550 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.509467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tthqn\" (UniqueName: \"kubernetes.io/projected/fa654b83-ef38-416c-a8ee-a65d4bede873-kube-api-access-tthqn\") pod \"authorino-58cd955675-x49x5\" (UID: \"fa654b83-ef38-416c-a8ee-a65d4bede873\") " pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:49.520354 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.520324    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tthqn\" (UniqueName: \"kubernetes.io/projected/fa654b83-ef38-416c-a8ee-a65d4bede873-kube-api-access-tthqn\") pod \"authorino-58cd955675-x49x5\" (UID: \"fa654b83-ef38-416c-a8ee-a65d4bede873\") " pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:49.549349 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.549325    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:49.699127 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:49.699084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-x49x5"]
Jul 07 00:03:49.703664 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:03:49.703638    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa654b83_ef38_416c_a8ee_a65d4bede873.slice/crio-55421b7d405ef669f03c36e35572691ef46c8dbbb5829dd5ac36131eb65a038e WatchSource:0}: Error finding container 55421b7d405ef669f03c36e35572691ef46c8dbbb5829dd5ac36131eb65a038e: Status 404 returned error can't find the container with id 55421b7d405ef669f03c36e35572691ef46c8dbbb5829dd5ac36131eb65a038e
Jul 07 00:03:50.335765 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:50.335702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x49x5" event={"ID":"fa654b83-ef38-416c-a8ee-a65d4bede873","Type":"ContainerStarted","Data":"55421b7d405ef669f03c36e35572691ef46c8dbbb5829dd5ac36131eb65a038e"}
Jul 07 00:03:54.577220 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:54.577176    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-x49x5"]
Jul 07 00:03:56.368706 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.368662    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x49x5" event={"ID":"fa654b83-ef38-416c-a8ee-a65d4bede873","Type":"ContainerStarted","Data":"5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420"}
Jul 07 00:03:56.369149 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.368685    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-x49x5" podUID="fa654b83-ef38-416c-a8ee-a65d4bede873" containerName="authorino" containerID="cri-o://5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420" gracePeriod=30
Jul 07 00:03:56.370320 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.370275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl" event={"ID":"0e952cae-6586-4e1a-8ef2-e1837eb33fa9","Type":"ContainerStarted","Data":"e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7"}
Jul 07 00:03:56.370474 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.370377    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:03:56.397975 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.397934    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-x49x5" podStartSLOduration=1.613551931 podStartE2EDuration="7.397920201s" totalImagesPullingTime="5.78436827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:56.395504861 +0000 UTC m=+460.760808619" watchObservedRunningTime="2026-07-07 00:03:56.397920201 +0000 UTC m=+460.763223945"
Jul 07 00:03:56.611116 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.611089    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:56.645609 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.645509    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl" podStartSLOduration=2.183710479 podStartE2EDuration="8.64549319s" totalImagesPullingTime="6.461782711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:56.421711397 +0000 UTC m=+460.787015142" watchObservedRunningTime="2026-07-07 00:03:56.64549319 +0000 UTC m=+461.010796996"
Jul 07 00:03:56.676764 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.676738    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tthqn\" (UniqueName: \"kubernetes.io/projected/fa654b83-ef38-416c-a8ee-a65d4bede873-kube-api-access-tthqn\") pod \"fa654b83-ef38-416c-a8ee-a65d4bede873\" (UID: \"fa654b83-ef38-416c-a8ee-a65d4bede873\") "
Jul 07 00:03:56.678991 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.678969    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa654b83-ef38-416c-a8ee-a65d4bede873-kube-api-access-tthqn" (OuterVolumeSpecName: "kube-api-access-tthqn") pod "fa654b83-ef38-416c-a8ee-a65d4bede873" (UID: "fa654b83-ef38-416c-a8ee-a65d4bede873"). InnerVolumeSpecName "kube-api-access-tthqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:03:56.778046 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:56.778017    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tthqn\" (UniqueName: \"kubernetes.io/projected/fa654b83-ef38-416c-a8ee-a65d4bede873-kube-api-access-tthqn\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:03:57.375854 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.375824    2583 generic.go:356] "Generic (PLEG): container finished" podID="fa654b83-ef38-416c-a8ee-a65d4bede873" containerID="5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420" exitCode=0
Jul 07 00:03:57.376309 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.375870    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x49x5"
Jul 07 00:03:57.376309 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.375919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x49x5" event={"ID":"fa654b83-ef38-416c-a8ee-a65d4bede873","Type":"ContainerDied","Data":"5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420"}
Jul 07 00:03:57.376309 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.375967    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x49x5" event={"ID":"fa654b83-ef38-416c-a8ee-a65d4bede873","Type":"ContainerDied","Data":"55421b7d405ef669f03c36e35572691ef46c8dbbb5829dd5ac36131eb65a038e"}
Jul 07 00:03:57.376309 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.375994    2583 scope.go:117] "RemoveContainer" containerID="5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420"
Jul 07 00:03:57.385110 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.385094    2583 scope.go:117] "RemoveContainer" containerID="5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420"
Jul 07 00:03:57.385362 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:03:57.385342    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420\": container with ID starting with 5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420 not found: ID does not exist" containerID="5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420"
Jul 07 00:03:57.385400 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.385373    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420"} err="failed to get container status \"5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420\": rpc error: code = NotFound desc = could not find container \"5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420\": container with ID starting with 5985a1f407070e32caa3079646435037b16f9b2833d4c7019459a182017c1420 not found: ID does not exist"
Jul 07 00:03:57.403483 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.403458    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-x49x5"]
Jul 07 00:03:57.407749 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:57.407726    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-x49x5"]
Jul 07 00:03:58.157472 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:03:58.157443    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa654b83-ef38-416c-a8ee-a65d4bede873" path="/var/lib/kubelet/pods/fa654b83-ef38-416c-a8ee-a65d4bede873/volumes"
Jul 07 00:04:05.579122 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:05.579088    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:04:05.579574 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:05.579275    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl" podUID="0e952cae-6586-4e1a-8ef2-e1837eb33fa9" containerName="limitador" containerID="cri-o://e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7" gracePeriod=30
Jul 07 00:04:05.581213 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:05.581194    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:04:06.118900 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.118876    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:04:06.158327 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.158235    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-config-file\") pod \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") "
Jul 07 00:04:06.158327 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.158284    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4rmt\" (UniqueName: \"kubernetes.io/projected/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-kube-api-access-l4rmt\") pod \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\" (UID: \"0e952cae-6586-4e1a-8ef2-e1837eb33fa9\") "
Jul 07 00:04:06.158659 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.158635    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-config-file" (OuterVolumeSpecName: "config-file") pod "0e952cae-6586-4e1a-8ef2-e1837eb33fa9" (UID: "0e952cae-6586-4e1a-8ef2-e1837eb33fa9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:04:06.160455 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.160438    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-kube-api-access-l4rmt" (OuterVolumeSpecName: "kube-api-access-l4rmt") pod "0e952cae-6586-4e1a-8ef2-e1837eb33fa9" (UID: "0e952cae-6586-4e1a-8ef2-e1837eb33fa9"). InnerVolumeSpecName "kube-api-access-l4rmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:06.259351 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.259313    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4rmt\" (UniqueName: \"kubernetes.io/projected/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-kube-api-access-l4rmt\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:04:06.259351 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.259349    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0e952cae-6586-4e1a-8ef2-e1837eb33fa9-config-file\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:04:06.417171 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.417090    2583 generic.go:356] "Generic (PLEG): container finished" podID="0e952cae-6586-4e1a-8ef2-e1837eb33fa9" containerID="e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7" exitCode=0
Jul 07 00:04:06.417171 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.417140    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl"
Jul 07 00:04:06.417171 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.417166    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl" event={"ID":"0e952cae-6586-4e1a-8ef2-e1837eb33fa9","Type":"ContainerDied","Data":"e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7"}
Jul 07 00:04:06.417414 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.417208    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qrprl" event={"ID":"0e952cae-6586-4e1a-8ef2-e1837eb33fa9","Type":"ContainerDied","Data":"8c17dedf6faafabdbd66ab915c8ef3b7fcf11eab01f25637ef840ff059023990"}
Jul 07 00:04:06.417414 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.417229    2583 scope.go:117] "RemoveContainer" containerID="e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7"
Jul 07 00:04:06.426642 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.426551    2583 scope.go:117] "RemoveContainer" containerID="e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7"
Jul 07 00:04:06.426891 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:04:06.426870    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7\": container with ID starting with e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7 not found: ID does not exist" containerID="e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7"
Jul 07 00:04:06.426983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.426897    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7"} err="failed to get container status \"e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7\": rpc error: code = NotFound desc = could not find container \"e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7\": container with ID starting with e83b87e21b5b5868979213d87f965eb8f011c53be664940fb9ac63580d2104b7 not found: ID does not exist"
Jul 07 00:04:06.438508 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.438485    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:04:06.444166 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:06.444144    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qrprl"]
Jul 07 00:04:08.157441 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:08.157411    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e952cae-6586-4e1a-8ef2-e1837eb33fa9" path="/var/lib/kubelet/pods/0e952cae-6586-4e1a-8ef2-e1837eb33fa9/volumes"
Jul 07 00:04:09.914476 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.914443    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-rrqrs"]
Jul 07 00:04:09.914976 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.914957    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e952cae-6586-4e1a-8ef2-e1837eb33fa9" containerName="limitador"
Jul 07 00:04:09.915027 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.914979    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e952cae-6586-4e1a-8ef2-e1837eb33fa9" containerName="limitador"
Jul 07 00:04:09.915027 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.914999    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa654b83-ef38-416c-a8ee-a65d4bede873" containerName="authorino"
Jul 07 00:04:09.915027 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.915006    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa654b83-ef38-416c-a8ee-a65d4bede873" containerName="authorino"
Jul 07 00:04:09.915113 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.915093    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa654b83-ef38-416c-a8ee-a65d4bede873" containerName="authorino"
Jul 07 00:04:09.915113 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.915103    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e952cae-6586-4e1a-8ef2-e1837eb33fa9" containerName="limitador"
Jul 07 00:04:09.917380 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.917365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:09.921434 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.921410    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7jk2n\""
Jul 07 00:04:09.921568 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.921540    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 07 00:04:09.933648 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.933626    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rrqrs"]
Jul 07 00:04:09.993309 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.993281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4qqk\" (UniqueName: \"kubernetes.io/projected/b7312506-02ca-4a85-a7d1-7cf7f169d17b-kube-api-access-p4qqk\") pod \"postgres-69fb4594cf-rrqrs\" (UID: \"b7312506-02ca-4a85-a7d1-7cf7f169d17b\") " pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:09.993433 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:09.993340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b7312506-02ca-4a85-a7d1-7cf7f169d17b-data\") pod \"postgres-69fb4594cf-rrqrs\" (UID: \"b7312506-02ca-4a85-a7d1-7cf7f169d17b\") " pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:10.094920 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.094885    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4qqk\" (UniqueName: \"kubernetes.io/projected/b7312506-02ca-4a85-a7d1-7cf7f169d17b-kube-api-access-p4qqk\") pod \"postgres-69fb4594cf-rrqrs\" (UID: \"b7312506-02ca-4a85-a7d1-7cf7f169d17b\") " pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:10.095076 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.094959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b7312506-02ca-4a85-a7d1-7cf7f169d17b-data\") pod \"postgres-69fb4594cf-rrqrs\" (UID: \"b7312506-02ca-4a85-a7d1-7cf7f169d17b\") " pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:10.095431 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.095413    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b7312506-02ca-4a85-a7d1-7cf7f169d17b-data\") pod \"postgres-69fb4594cf-rrqrs\" (UID: \"b7312506-02ca-4a85-a7d1-7cf7f169d17b\") " pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:10.107612 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.107587    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4qqk\" (UniqueName: \"kubernetes.io/projected/b7312506-02ca-4a85-a7d1-7cf7f169d17b-kube-api-access-p4qqk\") pod \"postgres-69fb4594cf-rrqrs\" (UID: \"b7312506-02ca-4a85-a7d1-7cf7f169d17b\") " pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:10.227126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.227047    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:10.362326 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.362298    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rrqrs"]
Jul 07 00:04:10.364807 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:04:10.364779    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7312506_02ca_4a85_a7d1_7cf7f169d17b.slice/crio-6038ad7750fb75f955c800b8203d20b1c3f67cedb5d771e1dafb0df14c4fe6ce WatchSource:0}: Error finding container 6038ad7750fb75f955c800b8203d20b1c3f67cedb5d771e1dafb0df14c4fe6ce: Status 404 returned error can't find the container with id 6038ad7750fb75f955c800b8203d20b1c3f67cedb5d771e1dafb0df14c4fe6ce
Jul 07 00:04:10.436637 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:10.436594    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rrqrs" event={"ID":"b7312506-02ca-4a85-a7d1-7cf7f169d17b","Type":"ContainerStarted","Data":"6038ad7750fb75f955c800b8203d20b1c3f67cedb5d771e1dafb0df14c4fe6ce"}
Jul 07 00:04:21.444049 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:21.444023    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 07 00:04:22.500491 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:22.500456    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rrqrs" event={"ID":"b7312506-02ca-4a85-a7d1-7cf7f169d17b","Type":"ContainerStarted","Data":"ca98b87deceecb75cb0c9bd1a5a22ad4d440cb74e5306c28536b64c5f037caae"}
Jul 07 00:04:22.500925 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:22.500564    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:04:22.518436 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:22.518386    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-rrqrs" podStartSLOduration=2.443587294 podStartE2EDuration="13.518372032s" totalImagesPullingTime="11.074784738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:22.517392291 +0000 UTC m=+486.882696036" watchObservedRunningTime="2026-07-07 00:04:22.518372032 +0000 UTC m=+486.883675778"
Jul 07 00:04:28.535741 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:04:28.535710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-rrqrs"
Jul 07 00:05:11.483182 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.483094    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9mpf7"]
Jul 07 00:05:11.485596 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.485572    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.488388 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.488363    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qc7mq\""
Jul 07 00:05:11.488513 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.488363    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 00:05:11.500171 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.500140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9mpf7"]
Jul 07 00:05:11.654746 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.654709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf168186-0bb9-44ca-aa36-1d4b3b11f1c8-plugins-config-volume\") pod \"payload-processing-58f86c7798-9mpf7\" (UID: \"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8\") " pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.654746 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.654746    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jkms\" (UniqueName: \"kubernetes.io/projected/cf168186-0bb9-44ca-aa36-1d4b3b11f1c8-kube-api-access-2jkms\") pod \"payload-processing-58f86c7798-9mpf7\" (UID: \"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8\") " pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.756294 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.756258    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf168186-0bb9-44ca-aa36-1d4b3b11f1c8-plugins-config-volume\") pod \"payload-processing-58f86c7798-9mpf7\" (UID: \"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8\") " pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.756294 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.756297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jkms\" (UniqueName: \"kubernetes.io/projected/cf168186-0bb9-44ca-aa36-1d4b3b11f1c8-kube-api-access-2jkms\") pod \"payload-processing-58f86c7798-9mpf7\" (UID: \"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8\") " pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.757006 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.756984    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf168186-0bb9-44ca-aa36-1d4b3b11f1c8-plugins-config-volume\") pod \"payload-processing-58f86c7798-9mpf7\" (UID: \"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8\") " pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.764962 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.764931    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jkms\" (UniqueName: \"kubernetes.io/projected/cf168186-0bb9-44ca-aa36-1d4b3b11f1c8-kube-api-access-2jkms\") pod \"payload-processing-58f86c7798-9mpf7\" (UID: \"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8\") " pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.795939 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.795901    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:11.933658 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:11.933630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9mpf7"]
Jul 07 00:05:11.936451 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:05:11.936423    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf168186_0bb9_44ca_aa36_1d4b3b11f1c8.slice/crio-284579e7a72d5f005ec0d8f19bda7abbe9ea1d3b548d9f37fec9a633ed25b49a WatchSource:0}: Error finding container 284579e7a72d5f005ec0d8f19bda7abbe9ea1d3b548d9f37fec9a633ed25b49a: Status 404 returned error can't find the container with id 284579e7a72d5f005ec0d8f19bda7abbe9ea1d3b548d9f37fec9a633ed25b49a
Jul 07 00:05:12.726657 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:12.726618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7" event={"ID":"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8","Type":"ContainerStarted","Data":"284579e7a72d5f005ec0d8f19bda7abbe9ea1d3b548d9f37fec9a633ed25b49a"}
Jul 07 00:05:14.737037 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:14.736997    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7" event={"ID":"cf168186-0bb9-44ca-aa36-1d4b3b11f1c8","Type":"ContainerStarted","Data":"661b86549bb498a0ac6ba27bae842b06541afa7158e447a7a5e0dad8626e7ce0"}
Jul 07 00:05:14.737393 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:14.737135    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:14.752818 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:14.752766    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7" podStartSLOduration=1.216267594 podStartE2EDuration="3.752751526s" totalImagesPullingTime="2.536483932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:14.751952844 +0000 UTC m=+539.117256591" watchObservedRunningTime="2026-07-07 00:05:14.752751526 +0000 UTC m=+539.118055271"
Jul 07 00:05:25.743718 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:25.743682    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-9mpf7"
Jul 07 00:05:31.265834 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.265792    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"]
Jul 07 00:05:31.274435 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.274406    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.277293 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.277269    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 00:05:31.277413 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.277277    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 07 00:05:31.278502 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.278478    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 00:05:31.278631 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.278579    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gq64g\""
Jul 07 00:05:31.282578 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.282546    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"]
Jul 07 00:05:31.332741 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.332707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.332898 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.332767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.332898 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.332850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.332898 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.332881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.333011 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.332919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plpq5\" (UniqueName: \"kubernetes.io/projected/7f34f6ad-7e13-4015-ada0-f27d7d44e534-kube-api-access-plpq5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.333011 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.332939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f34f6ad-7e13-4015-ada0-f27d7d44e534-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434259 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434457 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434457 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434310    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434457 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434335    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plpq5\" (UniqueName: \"kubernetes.io/projected/7f34f6ad-7e13-4015-ada0-f27d7d44e534-kube-api-access-plpq5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434457 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f34f6ad-7e13-4015-ada0-f27d7d44e534-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434457 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434814 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434724    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434814 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434786    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.434905 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.434817    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.436874 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.436854    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f34f6ad-7e13-4015-ada0-f27d7d44e534-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.437025 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.437007    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f34f6ad-7e13-4015-ada0-f27d7d44e534-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.446389 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.446365    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plpq5\" (UniqueName: \"kubernetes.io/projected/7f34f6ad-7e13-4015-ada0-f27d7d44e534-kube-api-access-plpq5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr\" (UID: \"7f34f6ad-7e13-4015-ada0-f27d7d44e534\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.586120 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.586036    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:31.722489 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.722466    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"]
Jul 07 00:05:31.724081 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:05:31.724042    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f34f6ad_7e13_4015_ada0_f27d7d44e534.slice/crio-03f2327c760c456e15efcdaae7f3039bdae08b5996c0191fe245f825d552d8ef WatchSource:0}: Error finding container 03f2327c760c456e15efcdaae7f3039bdae08b5996c0191fe245f825d552d8ef: Status 404 returned error can't find the container with id 03f2327c760c456e15efcdaae7f3039bdae08b5996c0191fe245f825d552d8ef
Jul 07 00:05:31.812736 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:31.812694    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr" event={"ID":"7f34f6ad-7e13-4015-ada0-f27d7d44e534","Type":"ContainerStarted","Data":"03f2327c760c456e15efcdaae7f3039bdae08b5996c0191fe245f825d552d8ef"}
Jul 07 00:05:32.821869 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:32.821828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr" event={"ID":"7f34f6ad-7e13-4015-ada0-f27d7d44e534","Type":"ContainerStarted","Data":"cd7aebccab8c0415f6493ed2a8e830314bf1cd27573d8cd2561a86c575405e39"}
Jul 07 00:05:32.842386 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:32.842329    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr" podStartSLOduration=0.837520905 podStartE2EDuration="1.842313077s" totalImagesPullingTime="1.004792172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:32.840646001 +0000 UTC m=+557.205949748" watchObservedRunningTime="2026-07-07 00:05:32.842313077 +0000 UTC m=+557.207616822"
Jul 07 00:05:41.586187 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:41.586130    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:41.586711 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:41.586213    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:41.598688 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:41.598658    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:41.881751 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:41.881659    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr"
Jul 07 00:05:42.480427 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.480390    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"]
Jul 07 00:05:42.482960 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.482939    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.505827 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.505795    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 07 00:05:42.521628 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.521582    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"]
Jul 07 00:05:42.534013 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.533978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd29d5c8-1c55-40ff-9708-fdabbe65327f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.534126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.534027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.534126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.534052    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.534198 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.534127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxhvx\" (UniqueName: \"kubernetes.io/projected/bd29d5c8-1c55-40ff-9708-fdabbe65327f-kube-api-access-sxhvx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.534198 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.534182    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.534264 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.534207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635013 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.634971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635013 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd29d5c8-1c55-40ff-9708-fdabbe65327f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635175    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxhvx\" (UniqueName: \"kubernetes.io/projected/bd29d5c8-1c55-40ff-9708-fdabbe65327f-kube-api-access-sxhvx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635405    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635464    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.635579 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.635486    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.637599 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.637578    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd29d5c8-1c55-40ff-9708-fdabbe65327f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.637875 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.637856    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd29d5c8-1c55-40ff-9708-fdabbe65327f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.670932 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.670891    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxhvx\" (UniqueName: \"kubernetes.io/projected/bd29d5c8-1c55-40ff-9708-fdabbe65327f-kube-api-access-sxhvx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-tdwk4\" (UID: \"bd29d5c8-1c55-40ff-9708-fdabbe65327f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.793039 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.792995    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:42.948891 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:42.948865    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"]
Jul 07 00:05:42.951504 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:05:42.951479    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd29d5c8_1c55_40ff_9708_fdabbe65327f.slice/crio-eb1486daa4b548618869d5c42a38fccbdcc1f430d1b3f17d58f43ba88e9151c0 WatchSource:0}: Error finding container eb1486daa4b548618869d5c42a38fccbdcc1f430d1b3f17d58f43ba88e9151c0: Status 404 returned error can't find the container with id eb1486daa4b548618869d5c42a38fccbdcc1f430d1b3f17d58f43ba88e9151c0
Jul 07 00:05:43.874886 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:43.874850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4" event={"ID":"bd29d5c8-1c55-40ff-9708-fdabbe65327f","Type":"ContainerStarted","Data":"ff4032480460126714ecf064f4c878fe5c5350e39ca7b3f09a05cba0967455d4"}
Jul 07 00:05:43.875368 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:43.874900    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4" event={"ID":"bd29d5c8-1c55-40ff-9708-fdabbe65327f","Type":"ContainerStarted","Data":"eb1486daa4b548618869d5c42a38fccbdcc1f430d1b3f17d58f43ba88e9151c0"}
Jul 07 00:05:43.894652 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:43.894601    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4" podStartSLOduration=1.682776565 podStartE2EDuration="1.894583754s" totalImagesPullingTime="211.807189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:43.893334402 +0000 UTC m=+568.258638148" watchObservedRunningTime="2026-07-07 00:05:43.894583754 +0000 UTC m=+568.259887500"
Jul 07 00:05:52.793551 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:52.793491    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:52.793957 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:52.793580    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:52.806129 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:52.806098    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:52.927085 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:52.927057    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-tdwk4"
Jul 07 00:05:59.357252 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.357207    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"]
Jul 07 00:05:59.360012 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.359984    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.362641 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.362617    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 00:05:59.373879 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.373848    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"]
Jul 07 00:05:59.503835 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.503790    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.503835 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.503840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.504057 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.503863    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.504057 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.503945    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.504057 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.503966    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8rhf\" (UniqueName: \"kubernetes.io/projected/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-kube-api-access-w8rhf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.504057 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.504037    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.605599 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.605519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.605782 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.605611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8rhf\" (UniqueName: \"kubernetes.io/projected/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-kube-api-access-w8rhf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.605782 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.605658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.605782 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.605698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.605782 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.605734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.605782 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.605767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.606027 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.606000    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.606102 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.606076    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.606242 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.606105    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.608268 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.608182    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.608424 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.608404    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.617471 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.617438    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8rhf\" (UniqueName: \"kubernetes.io/projected/d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1-kube-api-access-w8rhf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx\" (UID: \"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.670035 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.669987    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:05:59.817861 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.817825    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"]
Jul 07 00:05:59.820970 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:05:59.820933    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd79b616e_ed43_4f8e_b0b4_8a46ebf46ff1.slice/crio-405a572043a61f57d07c1562d7fe1cb7f063975f9a635c0255a8c4bb73830b60 WatchSource:0}: Error finding container 405a572043a61f57d07c1562d7fe1cb7f063975f9a635c0255a8c4bb73830b60: Status 404 returned error can't find the container with id 405a572043a61f57d07c1562d7fe1cb7f063975f9a635c0255a8c4bb73830b60
Jul 07 00:05:59.952124 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:05:59.952028    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx" event={"ID":"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1","Type":"ContainerStarted","Data":"405a572043a61f57d07c1562d7fe1cb7f063975f9a635c0255a8c4bb73830b60"}
Jul 07 00:06:00.958205 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:00.958164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx" event={"ID":"d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1","Type":"ContainerStarted","Data":"914b3432e01a1e5c79880d4e0212b23165474142d5f3c1c0e4cb2dde48f91d54"}
Jul 07 00:06:00.976500 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:00.976437    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx" podStartSLOduration=1.7695315329999999 podStartE2EDuration="1.976417145s" totalImagesPullingTime="206.885612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:00.975272952 +0000 UTC m=+585.340576698" watchObservedRunningTime="2026-07-07 00:06:00.976417145 +0000 UTC m=+585.341720890"
Jul 07 00:06:09.670343 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:09.670282    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:06:09.670343 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:09.670367    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:06:09.683053 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:09.683022    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:06:10.014015 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:10.013977    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx"
Jul 07 00:06:16.070709 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:16.070678    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:06:16.073131 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:06:16.073101    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:11:16.108980 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:11:16.108895    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:11:16.109589 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:11:16.109107    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:16:16.138837 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:16:16.138806    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:16:16.139680 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:16:16.139659    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:21:16.168453 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:21:16.168422    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:21:16.170442 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:21:16.170420    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:23:15.942804 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.942715    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"]
Jul 07 00:23:15.946392 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.946369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:15.950193 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.950155    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"e2e-lifecycle-model-bf334991-kserve-self-signed-certs\""
Jul 07 00:23:15.950344 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.950203    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"openshift-service-ca.crt\""
Jul 07 00:23:15.950344 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.950176    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"kube-root-ca.crt\""
Jul 07 00:23:15.950344 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.950176    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"default-dockercfg-zdcrd\""
Jul 07 00:23:15.957984 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:15.957957    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"]
Jul 07 00:23:16.096749 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.096712    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd949776-b347-4505-9a94-a937ef4453f4-tls-certs\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.096932 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.096769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-model-cache\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.096932 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.096826    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-home\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.096932 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.096843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp2tc\" (UniqueName: \"kubernetes.io/projected/bd949776-b347-4505-9a94-a937ef4453f4-kube-api-access-hp2tc\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.096932 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.096914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-dshm\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.097069 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.096977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-tmp-dir\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198255 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-model-cache\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198255 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-home\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198255 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp2tc\" (UniqueName: \"kubernetes.io/projected/bd949776-b347-4505-9a94-a937ef4453f4-kube-api-access-hp2tc\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198518 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-dshm\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198518 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198303    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-tmp-dir\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198518 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198339    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd949776-b347-4505-9a94-a937ef4453f4-tls-certs\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198711 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198666    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-home\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198711 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198696    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-model-cache\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.198811 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.198721    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-tmp-dir\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.200837 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.200809    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"e2e-lifecycle-model-bf334991-kserve-self-signed-certs\""
Jul 07 00:23:16.200989 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.200890    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-dshm\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.206691 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.206660    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"kube-root-ca.crt\""
Jul 07 00:23:16.211152 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.211127    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd949776-b347-4505-9a94-a937ef4453f4-tls-certs\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.217561 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.217513    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"openshift-service-ca.crt\""
Jul 07 00:23:16.228380 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.228344    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp2tc\" (UniqueName: \"kubernetes.io/projected/bd949776-b347-4505-9a94-a937ef4453f4-kube-api-access-hp2tc\") pod \"e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") " pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.260170 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.260135    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bf334991\"/\"default-dockercfg-zdcrd\""
Jul 07 00:23:16.268412 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.268388    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:16.407445 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.407380    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"]
Jul 07 00:23:16.409774 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:23:16.409747    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd949776_b347_4505_9a94_a937ef4453f4.slice/crio-f5cb320d59ae1cb10cd0a3b58752d915e4eec8f64cfe2c42f5655dc59a480d63 WatchSource:0}: Error finding container f5cb320d59ae1cb10cd0a3b58752d915e4eec8f64cfe2c42f5655dc59a480d63: Status 404 returned error can't find the container with id f5cb320d59ae1cb10cd0a3b58752d915e4eec8f64cfe2c42f5655dc59a480d63
Jul 07 00:23:16.847369 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.847336    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq" event={"ID":"bd949776-b347-4505-9a94-a937ef4453f4","Type":"ContainerStarted","Data":"709b3428c1a518e211121861638dd2dcf71a2aa3320602f93ac576ad5695963f"}
Jul 07 00:23:16.847514 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.847378    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq" event={"ID":"bd949776-b347-4505-9a94-a937ef4453f4","Type":"ContainerStarted","Data":"f5cb320d59ae1cb10cd0a3b58752d915e4eec8f64cfe2c42f5655dc59a480d63"}
Jul 07 00:23:16.866262 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:16.866210    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq" podStartSLOduration=1.866194401 podStartE2EDuration="1.866194401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:16.865041394 +0000 UTC m=+1621.230345142" watchObservedRunningTime="2026-07-07 00:23:16.866194401 +0000 UTC m=+1621.231498147"
Jul 07 00:23:26.269274 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:26.269233    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:26.269692 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:26.269309    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:26.282025 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:26.282001    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:26.907254 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:26.907218    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:23:52.836450 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:23:52.836413    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"]
Jul 07 00:24:07.857881 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:07.857814    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq" podUID="bd949776-b347-4505-9a94-a937ef4453f4" containerName="main" containerID="cri-o://709b3428c1a518e211121861638dd2dcf71a2aa3320602f93ac576ad5695963f" gracePeriod=45
Jul 07 00:24:08.087471 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.087436    2583 generic.go:356] "Generic (PLEG): container finished" podID="bd949776-b347-4505-9a94-a937ef4453f4" containerID="709b3428c1a518e211121861638dd2dcf71a2aa3320602f93ac576ad5695963f" exitCode=0
Jul 07 00:24:08.087662 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.087508    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq" event={"ID":"bd949776-b347-4505-9a94-a937ef4453f4","Type":"ContainerDied","Data":"709b3428c1a518e211121861638dd2dcf71a2aa3320602f93ac576ad5695963f"}
Jul 07 00:24:08.111310 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.111249    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:24:08.203171 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203134    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-model-cache\") pod \"bd949776-b347-4505-9a94-a937ef4453f4\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") "
Jul 07 00:24:08.203367 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203201    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd949776-b347-4505-9a94-a937ef4453f4-tls-certs\") pod \"bd949776-b347-4505-9a94-a937ef4453f4\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") "
Jul 07 00:24:08.203367 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203261    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp2tc\" (UniqueName: \"kubernetes.io/projected/bd949776-b347-4505-9a94-a937ef4453f4-kube-api-access-hp2tc\") pod \"bd949776-b347-4505-9a94-a937ef4453f4\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") "
Jul 07 00:24:08.203367 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203282    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-tmp-dir\") pod \"bd949776-b347-4505-9a94-a937ef4453f4\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") "
Jul 07 00:24:08.203367 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203319    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-dshm\") pod \"bd949776-b347-4505-9a94-a937ef4453f4\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") "
Jul 07 00:24:08.203616 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203374    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-home\") pod \"bd949776-b347-4505-9a94-a937ef4453f4\" (UID: \"bd949776-b347-4505-9a94-a937ef4453f4\") "
Jul 07 00:24:08.203616 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203478    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd949776-b347-4505-9a94-a937ef4453f4" (UID: "bd949776-b347-4505-9a94-a937ef4453f4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:08.203721 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203657    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd949776-b347-4505-9a94-a937ef4453f4" (UID: "bd949776-b347-4505-9a94-a937ef4453f4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:08.203986 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.203958    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-home" (OuterVolumeSpecName: "home") pod "bd949776-b347-4505-9a94-a937ef4453f4" (UID: "bd949776-b347-4505-9a94-a937ef4453f4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:08.204547 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.204507    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.204824 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.204556    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.204824 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.204569    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.205909 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.205877    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-dshm" (OuterVolumeSpecName: "dshm") pod "bd949776-b347-4505-9a94-a937ef4453f4" (UID: "bd949776-b347-4505-9a94-a937ef4453f4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:08.206000 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.205921    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd949776-b347-4505-9a94-a937ef4453f4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd949776-b347-4505-9a94-a937ef4453f4" (UID: "bd949776-b347-4505-9a94-a937ef4453f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:24:08.206115 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.206098    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd949776-b347-4505-9a94-a937ef4453f4-kube-api-access-hp2tc" (OuterVolumeSpecName: "kube-api-access-hp2tc") pod "bd949776-b347-4505-9a94-a937ef4453f4" (UID: "bd949776-b347-4505-9a94-a937ef4453f4"). InnerVolumeSpecName "kube-api-access-hp2tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:08.305219 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.305175    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd949776-b347-4505-9a94-a937ef4453f4-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.305219 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.305214    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp2tc\" (UniqueName: \"kubernetes.io/projected/bd949776-b347-4505-9a94-a937ef4453f4-kube-api-access-hp2tc\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.305219 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:08.305223    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd949776-b347-4505-9a94-a937ef4453f4-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:24:09.093828 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:09.093797    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"
Jul 07 00:24:09.094301 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:09.093823    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq" event={"ID":"bd949776-b347-4505-9a94-a937ef4453f4","Type":"ContainerDied","Data":"f5cb320d59ae1cb10cd0a3b58752d915e4eec8f64cfe2c42f5655dc59a480d63"}
Jul 07 00:24:09.094301 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:09.093874    2583 scope.go:117] "RemoveContainer" containerID="709b3428c1a518e211121861638dd2dcf71a2aa3320602f93ac576ad5695963f"
Jul 07 00:24:09.116985 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:09.116944    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"]
Jul 07 00:24:09.121380 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:09.121350    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-bf334991/e2e-lifecycle-model-bf334991-kserve-7844f74d7-58rcq"]
Jul 07 00:24:10.157864 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:24:10.157832    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd949776-b347-4505-9a94-a937ef4453f4" path="/var/lib/kubelet/pods/bd949776-b347-4505-9a94-a937ef4453f4/volumes"
Jul 07 00:26:09.867853 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.867755    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"]
Jul 07 00:26:09.868432 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.868186    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd949776-b347-4505-9a94-a937ef4453f4" containerName="main"
Jul 07 00:26:09.868432 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.868199    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd949776-b347-4505-9a94-a937ef4453f4" containerName="main"
Jul 07 00:26:09.868432 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.868264    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd949776-b347-4505-9a94-a937ef4453f4" containerName="main"
Jul 07 00:26:09.871015 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.870995    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:09.875456 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.875432    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1e462f\"/\"openshift-service-ca.crt\""
Jul 07 00:26:09.875598 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.875541    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1e462f\"/\"test-model-1e462f-kserve-self-signed-certs\""
Jul 07 00:26:09.876659 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.876640    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1e462f\"/\"default-dockercfg-sc7lj\""
Jul 07 00:26:09.876768 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.876677    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1e462f\"/\"kube-root-ca.crt\""
Jul 07 00:26:09.884426 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.884403    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"]
Jul 07 00:26:09.927125 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.927094    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-home\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:09.927289 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.927175    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-dshm\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:09.927289 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.927196    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-model-cache\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:09.927289 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.927217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-tmp-dir\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:09.927289 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.927248    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvn9r\" (UniqueName: \"kubernetes.io/projected/f79332eb-b390-4229-8d4c-a865aecc40ba-kube-api-access-jvn9r\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:09.927289 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:09.927280    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79332eb-b390-4229-8d4c-a865aecc40ba-tls-certs\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.028653 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.028616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn9r\" (UniqueName: \"kubernetes.io/projected/f79332eb-b390-4229-8d4c-a865aecc40ba-kube-api-access-jvn9r\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.028848 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.028670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79332eb-b390-4229-8d4c-a865aecc40ba-tls-certs\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.028848 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.028715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-home\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.028848 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.028793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-dshm\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.028848 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.028817    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-model-cache\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.028848 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.028837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-tmp-dir\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.029335 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.029301    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-model-cache\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.029424 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.029343    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-tmp-dir\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.029424 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.029368    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-home\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.031176 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.031151    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-dshm\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.031480 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.031463    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79332eb-b390-4229-8d4c-a865aecc40ba-tls-certs\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.044239 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.044219    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvn9r\" (UniqueName: \"kubernetes.io/projected/f79332eb-b390-4229-8d4c-a865aecc40ba-kube-api-access-jvn9r\") pod \"test-model-1e462f-kserve-74698d8f89-dbb8c\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") " pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.180623 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.180513    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:10.321697 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.321666    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"]
Jul 07 00:26:10.323646 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:26:10.323622    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf79332eb_b390_4229_8d4c_a865aecc40ba.slice/crio-a94ae6bd3a2d98b81d25666d9890f8697704b6dbdd39c30807c5e729da0a7b21 WatchSource:0}: Error finding container a94ae6bd3a2d98b81d25666d9890f8697704b6dbdd39c30807c5e729da0a7b21: Status 404 returned error can't find the container with id a94ae6bd3a2d98b81d25666d9890f8697704b6dbdd39c30807c5e729da0a7b21
Jul 07 00:26:10.677426 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.677381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c" event={"ID":"f79332eb-b390-4229-8d4c-a865aecc40ba","Type":"ContainerStarted","Data":"853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8"}
Jul 07 00:26:10.677426 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.677426    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c" event={"ID":"f79332eb-b390-4229-8d4c-a865aecc40ba","Type":"ContainerStarted","Data":"a94ae6bd3a2d98b81d25666d9890f8697704b6dbdd39c30807c5e729da0a7b21"}
Jul 07 00:26:10.704855 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:10.704791    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c" podStartSLOduration=1.704771505 podStartE2EDuration="1.704771505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:10.703582727 +0000 UTC m=+1795.068886464" watchObservedRunningTime="2026-07-07 00:26:10.704771505 +0000 UTC m=+1795.070075250"
Jul 07 00:26:16.197495 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:16.197467    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:26:16.199235 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:16.199206    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:26:20.181514 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:20.181470    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:20.181514 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:20.181522    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:20.194121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:20.194094    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:20.736179 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:20.736142    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:26:27.752107 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.752071    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"]
Jul 07 00:26:27.755883 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.755859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.759930 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.759901    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-58927c\"/\"kube-root-ca.crt\""
Jul 07 00:26:27.759930 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.759917    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-58927c\"/\"default-dockercfg-shg6t\""
Jul 07 00:26:27.760142 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.759932    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-58927c\"/\"test-model-58927c-kserve-self-signed-certs\""
Jul 07 00:26:27.760142 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.759936    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-58927c\"/\"openshift-service-ca.crt\""
Jul 07 00:26:27.765490 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.765465    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"]
Jul 07 00:26:27.897265 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.897226    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zstfm\" (UniqueName: \"kubernetes.io/projected/9f94726d-e193-41e4-a3c3-d5ef409af8fc-kube-api-access-zstfm\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.897265 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.897269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-dshm\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.897475 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.897301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-home\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.897475 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.897377    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tmp-dir\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.897475 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.897459    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-model-cache\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.897605 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.897493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tls-certs\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.998842 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.998792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tmp-dir\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999048 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.998865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-model-cache\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999048 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.998893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tls-certs\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999048 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.998948    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstfm\" (UniqueName: \"kubernetes.io/projected/9f94726d-e193-41e4-a3c3-d5ef409af8fc-kube-api-access-zstfm\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999048 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.998982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-dshm\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999048 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.999021    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-home\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999314 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.999264    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-model-cache\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999314 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.999297    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tmp-dir\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:27.999418 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:27.999374    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-home\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:28.001452 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.001426    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-dshm\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:28.001614 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.001578    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tls-certs\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:28.014973 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.014906    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zstfm\" (UniqueName: \"kubernetes.io/projected/9f94726d-e193-41e4-a3c3-d5ef409af8fc-kube-api-access-zstfm\") pod \"test-model-58927c-kserve-6c8688dc97-hmbc7\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") " pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:28.066642 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.066604    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:28.208956 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.208897    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"]
Jul 07 00:26:28.211062 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:26:28.211032    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f94726d_e193_41e4_a3c3_d5ef409af8fc.slice/crio-7560462a71192b354dbef3bc102b8070ec6dad8f7c65c0f26818d23960ae3998 WatchSource:0}: Error finding container 7560462a71192b354dbef3bc102b8070ec6dad8f7c65c0f26818d23960ae3998: Status 404 returned error can't find the container with id 7560462a71192b354dbef3bc102b8070ec6dad8f7c65c0f26818d23960ae3998
Jul 07 00:26:28.764182 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.764139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7" event={"ID":"9f94726d-e193-41e4-a3c3-d5ef409af8fc","Type":"ContainerStarted","Data":"abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95"}
Jul 07 00:26:28.764692 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.764194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7" event={"ID":"9f94726d-e193-41e4-a3c3-d5ef409af8fc","Type":"ContainerStarted","Data":"7560462a71192b354dbef3bc102b8070ec6dad8f7c65c0f26818d23960ae3998"}
Jul 07 00:26:28.802001 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:28.801932    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7" podStartSLOduration=1.801916398 podStartE2EDuration="1.801916398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:28.800555208 +0000 UTC m=+1813.165858952" watchObservedRunningTime="2026-07-07 00:26:28.801916398 +0000 UTC m=+1813.167220144"
Jul 07 00:26:38.066977 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:38.066931    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:38.066977 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:38.066983    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:38.079461 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:38.079433    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:26:38.826363 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:26:38.826333    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:27:04.742811 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:04.742769    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"]
Jul 07 00:27:05.689686 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:05.689649    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"]
Jul 07 00:27:19.763973 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:19.763908    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c" podUID="f79332eb-b390-4229-8d4c-a865aecc40ba" containerName="main" containerID="cri-o://853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8" gracePeriod=45
Jul 07 00:27:20.007176 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.007141    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:27:20.009981 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.009946    2583 generic.go:356] "Generic (PLEG): container finished" podID="f79332eb-b390-4229-8d4c-a865aecc40ba" containerID="853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8" exitCode=0
Jul 07 00:27:20.010175 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.009984    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"
Jul 07 00:27:20.010175 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.009986    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c" event={"ID":"f79332eb-b390-4229-8d4c-a865aecc40ba","Type":"ContainerDied","Data":"853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8"}
Jul 07 00:27:20.010175 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.010121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c" event={"ID":"f79332eb-b390-4229-8d4c-a865aecc40ba","Type":"ContainerDied","Data":"a94ae6bd3a2d98b81d25666d9890f8697704b6dbdd39c30807c5e729da0a7b21"}
Jul 07 00:27:20.010175 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.010163    2583 scope.go:117] "RemoveContainer" containerID="853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8"
Jul 07 00:27:20.017091 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017032    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-home\") pod \"f79332eb-b390-4229-8d4c-a865aecc40ba\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") "
Jul 07 00:27:20.017091 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017071    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-tmp-dir\") pod \"f79332eb-b390-4229-8d4c-a865aecc40ba\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") "
Jul 07 00:27:20.017262 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017104    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-dshm\") pod \"f79332eb-b390-4229-8d4c-a865aecc40ba\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") "
Jul 07 00:27:20.017262 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017143    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-model-cache\") pod \"f79332eb-b390-4229-8d4c-a865aecc40ba\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") "
Jul 07 00:27:20.017262 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017225    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvn9r\" (UniqueName: \"kubernetes.io/projected/f79332eb-b390-4229-8d4c-a865aecc40ba-kube-api-access-jvn9r\") pod \"f79332eb-b390-4229-8d4c-a865aecc40ba\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") "
Jul 07 00:27:20.017262 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017258    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79332eb-b390-4229-8d4c-a865aecc40ba-tls-certs\") pod \"f79332eb-b390-4229-8d4c-a865aecc40ba\" (UID: \"f79332eb-b390-4229-8d4c-a865aecc40ba\") "
Jul 07 00:27:20.017463 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017338    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f79332eb-b390-4229-8d4c-a865aecc40ba" (UID: "f79332eb-b390-4229-8d4c-a865aecc40ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:20.017463 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017349    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-home" (OuterVolumeSpecName: "home") pod "f79332eb-b390-4229-8d4c-a865aecc40ba" (UID: "f79332eb-b390-4229-8d4c-a865aecc40ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:20.017463 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017437    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "f79332eb-b390-4229-8d4c-a865aecc40ba" (UID: "f79332eb-b390-4229-8d4c-a865aecc40ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:20.017651 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017614    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:20.017651 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017631    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:20.017651 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.017645    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:20.019602 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.019585    2583 scope.go:117] "RemoveContainer" containerID="853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8"
Jul 07 00:27:20.019947 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:27:20.019927    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8\": container with ID starting with 853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8 not found: ID does not exist" containerID="853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8"
Jul 07 00:27:20.020035 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.019960    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8"} err="failed to get container status \"853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8\": rpc error: code = NotFound desc = could not find container \"853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8\": container with ID starting with 853e202580f60015c86550a77c7c6f25fee56015b677f2a949be8109b35c61e8 not found: ID does not exist"
Jul 07 00:27:20.020098 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.020077    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-dshm" (OuterVolumeSpecName: "dshm") pod "f79332eb-b390-4229-8d4c-a865aecc40ba" (UID: "f79332eb-b390-4229-8d4c-a865aecc40ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:20.020098 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.020085    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79332eb-b390-4229-8d4c-a865aecc40ba-kube-api-access-jvn9r" (OuterVolumeSpecName: "kube-api-access-jvn9r") pod "f79332eb-b390-4229-8d4c-a865aecc40ba" (UID: "f79332eb-b390-4229-8d4c-a865aecc40ba"). InnerVolumeSpecName "kube-api-access-jvn9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:27:20.020180 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.020159    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f79332eb-b390-4229-8d4c-a865aecc40ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f79332eb-b390-4229-8d4c-a865aecc40ba" (UID: "f79332eb-b390-4229-8d4c-a865aecc40ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:27:20.118194 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.118147    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvn9r\" (UniqueName: \"kubernetes.io/projected/f79332eb-b390-4229-8d4c-a865aecc40ba-kube-api-access-jvn9r\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:20.118194 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.118186    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79332eb-b390-4229-8d4c-a865aecc40ba-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:20.118194 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.118200    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79332eb-b390-4229-8d4c-a865aecc40ba-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:20.326197 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.326106    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"]
Jul 07 00:27:20.330903 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.330868    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1e462f/test-model-1e462f-kserve-74698d8f89-dbb8c"]
Jul 07 00:27:20.711200 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.711111    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7" podUID="9f94726d-e193-41e4-a3c3-d5ef409af8fc" containerName="main" containerID="cri-o://abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95" gracePeriod=45
Jul 07 00:27:20.962889 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:20.962826    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:27:21.015989 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.015955    2583 generic.go:356] "Generic (PLEG): container finished" podID="9f94726d-e193-41e4-a3c3-d5ef409af8fc" containerID="abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95" exitCode=0
Jul 07 00:27:21.016163 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.016006    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"
Jul 07 00:27:21.016163 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.016044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7" event={"ID":"9f94726d-e193-41e4-a3c3-d5ef409af8fc","Type":"ContainerDied","Data":"abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95"}
Jul 07 00:27:21.016163 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.016084    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7" event={"ID":"9f94726d-e193-41e4-a3c3-d5ef409af8fc","Type":"ContainerDied","Data":"7560462a71192b354dbef3bc102b8070ec6dad8f7c65c0f26818d23960ae3998"}
Jul 07 00:27:21.016163 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.016105    2583 scope.go:117] "RemoveContainer" containerID="abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95"
Jul 07 00:27:21.024836 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.024816    2583 scope.go:117] "RemoveContainer" containerID="abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95"
Jul 07 00:27:21.025088 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:27:21.025068    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95\": container with ID starting with abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95 not found: ID does not exist" containerID="abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95"
Jul 07 00:27:21.025130 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025100    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95"} err="failed to get container status \"abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95\": rpc error: code = NotFound desc = could not find container \"abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95\": container with ID starting with abd30a38f54aaa09096aa3d51adda248d900904daddf942c495e35cbd4105c95 not found: ID does not exist"
Jul 07 00:27:21.025186 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025173    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tls-certs\") pod \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") "
Jul 07 00:27:21.025249 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025231    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tmp-dir\") pod \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") "
Jul 07 00:27:21.025350 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025258    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zstfm\" (UniqueName: \"kubernetes.io/projected/9f94726d-e193-41e4-a3c3-d5ef409af8fc-kube-api-access-zstfm\") pod \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") "
Jul 07 00:27:21.025350 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025294    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-dshm\") pod \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") "
Jul 07 00:27:21.025350 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025324    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-model-cache\") pod \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") "
Jul 07 00:27:21.025506 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025370    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-home\") pod \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\" (UID: \"9f94726d-e193-41e4-a3c3-d5ef409af8fc\") "
Jul 07 00:27:21.025586 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025559    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f94726d-e193-41e4-a3c3-d5ef409af8fc" (UID: "9f94726d-e193-41e4-a3c3-d5ef409af8fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:21.025801 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025749    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-home" (OuterVolumeSpecName: "home") pod "9f94726d-e193-41e4-a3c3-d5ef409af8fc" (UID: "9f94726d-e193-41e4-a3c3-d5ef409af8fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:21.025801 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025768    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:21.025801 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.025762    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f94726d-e193-41e4-a3c3-d5ef409af8fc" (UID: "9f94726d-e193-41e4-a3c3-d5ef409af8fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:21.027574 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.027548    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f94726d-e193-41e4-a3c3-d5ef409af8fc" (UID: "9f94726d-e193-41e4-a3c3-d5ef409af8fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:27:21.027664 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.027607    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f94726d-e193-41e4-a3c3-d5ef409af8fc-kube-api-access-zstfm" (OuterVolumeSpecName: "kube-api-access-zstfm") pod "9f94726d-e193-41e4-a3c3-d5ef409af8fc" (UID: "9f94726d-e193-41e4-a3c3-d5ef409af8fc"). InnerVolumeSpecName "kube-api-access-zstfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:27:21.027664 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.027638    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-dshm" (OuterVolumeSpecName: "dshm") pod "9f94726d-e193-41e4-a3c3-d5ef409af8fc" (UID: "9f94726d-e193-41e4-a3c3-d5ef409af8fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:27:21.126983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.126941    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f94726d-e193-41e4-a3c3-d5ef409af8fc-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:21.126983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.126978    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zstfm\" (UniqueName: \"kubernetes.io/projected/9f94726d-e193-41e4-a3c3-d5ef409af8fc-kube-api-access-zstfm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:21.126983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.126987    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:21.127211 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.126997    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:21.127211 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.127006    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f94726d-e193-41e4-a3c3-d5ef409af8fc-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:27:21.337931 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.337903    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"]
Jul 07 00:27:21.341510 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:21.341486    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-58927c/test-model-58927c-kserve-6c8688dc97-hmbc7"]
Jul 07 00:27:22.157298 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:22.157262    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f94726d-e193-41e4-a3c3-d5ef409af8fc" path="/var/lib/kubelet/pods/9f94726d-e193-41e4-a3c3-d5ef409af8fc/volumes"
Jul 07 00:27:22.157707 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:27:22.157692    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f79332eb-b390-4229-8d4c-a865aecc40ba" path="/var/lib/kubelet/pods/f79332eb-b390-4229-8d4c-a865aecc40ba/volumes"
Jul 07 00:29:06.956537 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956443    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"]
Jul 07 00:29:06.956983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956868    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79332eb-b390-4229-8d4c-a865aecc40ba" containerName="main"
Jul 07 00:29:06.956983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956881    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79332eb-b390-4229-8d4c-a865aecc40ba" containerName="main"
Jul 07 00:29:06.956983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956889    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f94726d-e193-41e4-a3c3-d5ef409af8fc" containerName="main"
Jul 07 00:29:06.956983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956895    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f94726d-e193-41e4-a3c3-d5ef409af8fc" containerName="main"
Jul 07 00:29:06.956983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956959    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79332eb-b390-4229-8d4c-a865aecc40ba" containerName="main"
Jul 07 00:29:06.956983 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.956966    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f94726d-e193-41e4-a3c3-d5ef409af8fc" containerName="main"
Jul 07 00:29:06.960034 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.960013    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:06.963627 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.963600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5f1a0\"/\"default-dockercfg-mjdft\""
Jul 07 00:29:06.963627 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.963625    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5f1a0\"/\"auth-test-model-f5f1a0-kserve-self-signed-certs\""
Jul 07 00:29:06.963841 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.963708    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f5f1a0\"/\"kube-root-ca.crt\""
Jul 07 00:29:06.963841 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.963716    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f5f1a0\"/\"openshift-service-ca.crt\""
Jul 07 00:29:06.970716 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:06.970687    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"]
Jul 07 00:29:07.058491 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.058446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-dshm\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.058491 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.058498    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tmp-dir\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.058744 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.058585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxl8\" (UniqueName: \"kubernetes.io/projected/3814ce87-bdf5-4367-80a7-5e750f05a1d3-kube-api-access-gcxl8\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.058744 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.058628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tls-certs\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.058744 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.058656    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-home\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.058744 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.058687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-model-cache\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160084 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tls-certs\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160084 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-home\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160326 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160114    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-model-cache\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160326 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-dshm\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160326 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tmp-dir\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160326 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxl8\" (UniqueName: \"kubernetes.io/projected/3814ce87-bdf5-4367-80a7-5e750f05a1d3-kube-api-access-gcxl8\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160665 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160632    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-home\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160665 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160654    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tmp-dir\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.160829 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.160707    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-model-cache\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.162689 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.162669    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-dshm\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.162859 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.162842    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tls-certs\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.170310 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.170280    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxl8\" (UniqueName: \"kubernetes.io/projected/3814ce87-bdf5-4367-80a7-5e750f05a1d3-kube-api-access-gcxl8\") pod \"auth-test-model-f5f1a0-kserve-868f954b87-2dt5r\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") " pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.269610 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.269562    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:07.415164 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.415129    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"]
Jul 07 00:29:07.417898 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:29:07.417863    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3814ce87_bdf5_4367_80a7_5e750f05a1d3.slice/crio-4656f5150fc467cade8174d9caca2394c26c3e1eac1f0219845b3947b5dedb20 WatchSource:0}: Error finding container 4656f5150fc467cade8174d9caca2394c26c3e1eac1f0219845b3947b5dedb20: Status 404 returned error can't find the container with id 4656f5150fc467cade8174d9caca2394c26c3e1eac1f0219845b3947b5dedb20
Jul 07 00:29:07.524897 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.524806    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r" event={"ID":"3814ce87-bdf5-4367-80a7-5e750f05a1d3","Type":"ContainerStarted","Data":"b529137f50d6de52c12c282e06a7ba1b3ff72a385e2c6d3c5e23e5730d8c9459"}
Jul 07 00:29:07.524897 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.524849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r" event={"ID":"3814ce87-bdf5-4367-80a7-5e750f05a1d3","Type":"ContainerStarted","Data":"4656f5150fc467cade8174d9caca2394c26c3e1eac1f0219845b3947b5dedb20"}
Jul 07 00:29:07.544146 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:07.544089    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r" podStartSLOduration=1.5440722999999998 podStartE2EDuration="1.5440723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:07.542774808 +0000 UTC m=+1971.908078555" watchObservedRunningTime="2026-07-07 00:29:07.5440723 +0000 UTC m=+1971.909376036"
Jul 07 00:29:17.270308 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:17.270264    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:17.270741 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:17.270321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:17.282938 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:17.282907    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:17.590251 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:17.590153    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:29:39.363225 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.363183    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"]
Jul 07 00:29:39.367195 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.367165    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.369907 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.369875    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"auth-test-model-ead650-kserve-self-signed-certs\""
Jul 07 00:29:39.371027 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.371006    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"kube-root-ca.crt\""
Jul 07 00:29:39.371118 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.371097    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"default-dockercfg-6zdqv\""
Jul 07 00:29:39.371164 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.371097    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"openshift-service-ca.crt\""
Jul 07 00:29:39.379775 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.379742    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"]
Jul 07 00:29:39.448760 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.448719    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-dshm\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.448969 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.448775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tmp-dir\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.448969 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.448905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tls-certs\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.449090 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.448980    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmvhz\" (UniqueName: \"kubernetes.io/projected/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-kube-api-access-kmvhz\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.449090 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.449025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-model-cache\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.449090 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.449076    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-home\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.549593 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549560    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmvhz\" (UniqueName: \"kubernetes.io/projected/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-kube-api-access-kmvhz\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.549780 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549606    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-model-cache\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.549780 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549631    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-home\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.549780 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-dshm\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.549780 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tmp-dir\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.549780 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tls-certs\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.550016 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.549997    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-model-cache\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.550075 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.550057    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-home\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.550146 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.550112    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tmp-dir\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.552207 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.552187    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-dshm\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.552374 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.552357    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tls-certs\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.559468 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.559432    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmvhz\" (UniqueName: \"kubernetes.io/projected/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-kube-api-access-kmvhz\") pod \"auth-test-model-ead650-kserve-d69f6f44c-pngn8\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") " pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.678197 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.678096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:39.825516 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:39.825480    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"]
Jul 07 00:29:39.828727 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:29:39.828691    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cc1e299_35cc_40ab_83fd_454e7fb2c06f.slice/crio-9a793b823af81b5d81b97031b04433fa52abb9cb320dad30272d3f4d7bd7bf3a WatchSource:0}: Error finding container 9a793b823af81b5d81b97031b04433fa52abb9cb320dad30272d3f4d7bd7bf3a: Status 404 returned error can't find the container with id 9a793b823af81b5d81b97031b04433fa52abb9cb320dad30272d3f4d7bd7bf3a
Jul 07 00:29:40.697381 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:40.697333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8" event={"ID":"5cc1e299-35cc-40ab-83fd-454e7fb2c06f","Type":"ContainerStarted","Data":"f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a"}
Jul 07 00:29:40.697817 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:40.697390    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8" event={"ID":"5cc1e299-35cc-40ab-83fd-454e7fb2c06f","Type":"ContainerStarted","Data":"9a793b823af81b5d81b97031b04433fa52abb9cb320dad30272d3f4d7bd7bf3a"}
Jul 07 00:29:40.720048 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:40.719998    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8" podStartSLOduration=1.7199813449999999 podStartE2EDuration="1.719981345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:40.719927587 +0000 UTC m=+2005.085231343" watchObservedRunningTime="2026-07-07 00:29:40.719981345 +0000 UTC m=+2005.085285088"
Jul 07 00:29:49.678819 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:49.678770    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:49.679323 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:49.678851    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:49.691568 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:49.691512    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:29:49.754430 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:29:49.754403    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:31:16.228367 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:16.228338    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:31:16.230213 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:16.230189    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdsfs_36d5608a-97e8-4e15-8a53-bb223736daeb/ovn-acl-logging/0.log"
Jul 07 00:31:37.066496 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:37.066461    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"]
Jul 07 00:31:37.949506 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:37.949472    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"]
Jul 07 00:31:48.144312 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.144280    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"]
Jul 07 00:31:48.147668 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.147649    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.151197 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.151176    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5f1a0\"/\"sub-test-model-f5f1a0-kserve-self-signed-certs\""
Jul 07 00:31:48.157834 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.157808    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"]
Jul 07 00:31:48.303980 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.303941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-model-cache\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.304167 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.303987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-home\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.304167 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.304027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ae6f99-e84c-4be2-a468-31619158fe22-tls-certs\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.304167 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.304084    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-dshm\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.304167 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.304121    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-tmp-dir\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.304167 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.304159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjqxq\" (UniqueName: \"kubernetes.io/projected/80ae6f99-e84c-4be2-a468-31619158fe22-kube-api-access-fjqxq\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405194 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjqxq\" (UniqueName: \"kubernetes.io/projected/80ae6f99-e84c-4be2-a468-31619158fe22-kube-api-access-fjqxq\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405194 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-model-cache\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405404 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405203    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-home\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405404 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405248    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ae6f99-e84c-4be2-a468-31619158fe22-tls-certs\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405404 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-dshm\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405404 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-tmp-dir\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405657 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405635    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-home\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405730 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405706    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-model-cache\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.405795 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.405779    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-tmp-dir\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.407828 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.407809    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-dshm\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.407978 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.407959    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ae6f99-e84c-4be2-a468-31619158fe22-tls-certs\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.415092 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.415063    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjqxq\" (UniqueName: \"kubernetes.io/projected/80ae6f99-e84c-4be2-a468-31619158fe22-kube-api-access-fjqxq\") pod \"sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") " pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.457684 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.457649    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:48.600787 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:48.600760    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"]
Jul 07 00:31:48.602214 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:31:48.602188    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80ae6f99_e84c_4be2_a468_31619158fe22.slice/crio-00f8278a5239773221e35f38c40d4285f13a1ba40f167a4b45c35bc401e36858 WatchSource:0}: Error finding container 00f8278a5239773221e35f38c40d4285f13a1ba40f167a4b45c35bc401e36858: Status 404 returned error can't find the container with id 00f8278a5239773221e35f38c40d4285f13a1ba40f167a4b45c35bc401e36858
Jul 07 00:31:49.318730 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:49.318686    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7" event={"ID":"80ae6f99-e84c-4be2-a468-31619158fe22","Type":"ContainerStarted","Data":"5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201"}
Jul 07 00:31:49.318730 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:49.318730    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7" event={"ID":"80ae6f99-e84c-4be2-a468-31619158fe22","Type":"ContainerStarted","Data":"00f8278a5239773221e35f38c40d4285f13a1ba40f167a4b45c35bc401e36858"}
Jul 07 00:31:49.338381 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:49.338325    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7" podStartSLOduration=1.338310145 podStartE2EDuration="1.338310145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:31:49.336725546 +0000 UTC m=+2133.702029291" watchObservedRunningTime="2026-07-07 00:31:49.338310145 +0000 UTC m=+2133.703613890"
Jul 07 00:31:52.087100 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.087050    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r" podUID="3814ce87-bdf5-4367-80a7-5e750f05a1d3" containerName="main" containerID="cri-o://b529137f50d6de52c12c282e06a7ba1b3ff72a385e2c6d3c5e23e5730d8c9459" gracePeriod=45
Jul 07 00:31:52.335121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.334901    2583 generic.go:356] "Generic (PLEG): container finished" podID="3814ce87-bdf5-4367-80a7-5e750f05a1d3" containerID="b529137f50d6de52c12c282e06a7ba1b3ff72a385e2c6d3c5e23e5730d8c9459" exitCode=0
Jul 07 00:31:52.335121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.334974    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r" event={"ID":"3814ce87-bdf5-4367-80a7-5e750f05a1d3","Type":"ContainerDied","Data":"b529137f50d6de52c12c282e06a7ba1b3ff72a385e2c6d3c5e23e5730d8c9459"}
Jul 07 00:31:52.335121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.335019    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r" event={"ID":"3814ce87-bdf5-4367-80a7-5e750f05a1d3","Type":"ContainerDied","Data":"4656f5150fc467cade8174d9caca2394c26c3e1eac1f0219845b3947b5dedb20"}
Jul 07 00:31:52.335121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.335031    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4656f5150fc467cade8174d9caca2394c26c3e1eac1f0219845b3947b5dedb20"
Jul 07 00:31:52.351406 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.351330    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:31:52.447230 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447193    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tmp-dir\") pod \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") "
Jul 07 00:31:52.447230 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447236    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-home\") pod \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") "
Jul 07 00:31:52.447487 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447263    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcxl8\" (UniqueName: \"kubernetes.io/projected/3814ce87-bdf5-4367-80a7-5e750f05a1d3-kube-api-access-gcxl8\") pod \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") "
Jul 07 00:31:52.447487 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447286    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tls-certs\") pod \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") "
Jul 07 00:31:52.447487 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447347    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-model-cache\") pod \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") "
Jul 07 00:31:52.447487 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447382    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-dshm\") pod \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\" (UID: \"3814ce87-bdf5-4367-80a7-5e750f05a1d3\") "
Jul 07 00:31:52.447831 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447565    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-home" (OuterVolumeSpecName: "home") pod "3814ce87-bdf5-4367-80a7-5e750f05a1d3" (UID: "3814ce87-bdf5-4367-80a7-5e750f05a1d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:52.447831 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447550    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3814ce87-bdf5-4367-80a7-5e750f05a1d3" (UID: "3814ce87-bdf5-4367-80a7-5e750f05a1d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:52.447831 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447619    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "3814ce87-bdf5-4367-80a7-5e750f05a1d3" (UID: "3814ce87-bdf5-4367-80a7-5e750f05a1d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:52.447831 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447776    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:52.447831 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447788    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:52.447831 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.447797    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:52.449742 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.449716    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-dshm" (OuterVolumeSpecName: "dshm") pod "3814ce87-bdf5-4367-80a7-5e750f05a1d3" (UID: "3814ce87-bdf5-4367-80a7-5e750f05a1d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:52.449833 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.449777    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3814ce87-bdf5-4367-80a7-5e750f05a1d3-kube-api-access-gcxl8" (OuterVolumeSpecName: "kube-api-access-gcxl8") pod "3814ce87-bdf5-4367-80a7-5e750f05a1d3" (UID: "3814ce87-bdf5-4367-80a7-5e750f05a1d3"). InnerVolumeSpecName "kube-api-access-gcxl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:31:52.449884 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.449825    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3814ce87-bdf5-4367-80a7-5e750f05a1d3" (UID: "3814ce87-bdf5-4367-80a7-5e750f05a1d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:31:52.548536 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.548501    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcxl8\" (UniqueName: \"kubernetes.io/projected/3814ce87-bdf5-4367-80a7-5e750f05a1d3-kube-api-access-gcxl8\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:52.548705 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.548555    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3814ce87-bdf5-4367-80a7-5e750f05a1d3-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:52.548705 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.548568    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3814ce87-bdf5-4367-80a7-5e750f05a1d3-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:52.978549 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:52.978475    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8" podUID="5cc1e299-35cc-40ab-83fd-454e7fb2c06f" containerName="main" containerID="cri-o://f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a" gracePeriod=45
Jul 07 00:31:53.236923 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.236901    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:31:53.341424 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.341385    2583 generic.go:356] "Generic (PLEG): container finished" podID="5cc1e299-35cc-40ab-83fd-454e7fb2c06f" containerID="f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a" exitCode=0
Jul 07 00:31:53.341626 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.341433    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"
Jul 07 00:31:53.341626 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.341472    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8" event={"ID":"5cc1e299-35cc-40ab-83fd-454e7fb2c06f","Type":"ContainerDied","Data":"f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a"}
Jul 07 00:31:53.341626 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.341516    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8" event={"ID":"5cc1e299-35cc-40ab-83fd-454e7fb2c06f","Type":"ContainerDied","Data":"9a793b823af81b5d81b97031b04433fa52abb9cb320dad30272d3f4d7bd7bf3a"}
Jul 07 00:31:53.341626 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.341567    2583 scope.go:117] "RemoveContainer" containerID="f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a"
Jul 07 00:31:53.341626 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.341618    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"
Jul 07 00:31:53.350509 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.350493    2583 scope.go:117] "RemoveContainer" containerID="f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a"
Jul 07 00:31:53.350800 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:31:53.350781    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a\": container with ID starting with f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a not found: ID does not exist" containerID="f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a"
Jul 07 00:31:53.350858 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.350806    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a"} err="failed to get container status \"f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a\": rpc error: code = NotFound desc = could not find container \"f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a\": container with ID starting with f22ae08ef4e442ef6082d5fd0f17ebeb2eda97ce2b65cfd054a02109a0db546a not found: ID does not exist"
Jul 07 00:31:53.355977 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.355961    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-dshm\") pod \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") "
Jul 07 00:31:53.356027 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.355993    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmvhz\" (UniqueName: \"kubernetes.io/projected/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-kube-api-access-kmvhz\") pod \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") "
Jul 07 00:31:53.356062 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356051    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-home\") pod \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") "
Jul 07 00:31:53.356097 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356077    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tmp-dir\") pod \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") "
Jul 07 00:31:53.356143 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356121    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tls-certs\") pod \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") "
Jul 07 00:31:53.356233 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356217    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-model-cache\") pod \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\" (UID: \"5cc1e299-35cc-40ab-83fd-454e7fb2c06f\") "
Jul 07 00:31:53.356356 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356332    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-home" (OuterVolumeSpecName: "home") pod "5cc1e299-35cc-40ab-83fd-454e7fb2c06f" (UID: "5cc1e299-35cc-40ab-83fd-454e7fb2c06f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:53.356468 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356354    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5cc1e299-35cc-40ab-83fd-454e7fb2c06f" (UID: "5cc1e299-35cc-40ab-83fd-454e7fb2c06f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:53.356553 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356469    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-model-cache" (OuterVolumeSpecName: "model-cache") pod "5cc1e299-35cc-40ab-83fd-454e7fb2c06f" (UID: "5cc1e299-35cc-40ab-83fd-454e7fb2c06f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:53.356793 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356744    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:53.356847 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356799    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:53.356847 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.356813    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:53.358325 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.358300    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-dshm" (OuterVolumeSpecName: "dshm") pod "5cc1e299-35cc-40ab-83fd-454e7fb2c06f" (UID: "5cc1e299-35cc-40ab-83fd-454e7fb2c06f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:53.358325 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.358315    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5cc1e299-35cc-40ab-83fd-454e7fb2c06f" (UID: "5cc1e299-35cc-40ab-83fd-454e7fb2c06f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:31:53.358439 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.358362    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-kube-api-access-kmvhz" (OuterVolumeSpecName: "kube-api-access-kmvhz") pod "5cc1e299-35cc-40ab-83fd-454e7fb2c06f" (UID: "5cc1e299-35cc-40ab-83fd-454e7fb2c06f"). InnerVolumeSpecName "kube-api-access-kmvhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:31:53.362769 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.362749    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"]
Jul 07 00:31:53.368898 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.368874    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/auth-test-model-f5f1a0-kserve-868f954b87-2dt5r"]
Jul 07 00:31:53.457943 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.457903    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:53.457943 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.457936    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:53.457943 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.457948    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmvhz\" (UniqueName: \"kubernetes.io/projected/5cc1e299-35cc-40ab-83fd-454e7fb2c06f-kube-api-access-kmvhz\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:31:53.666970 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.666934    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"]
Jul 07 00:31:53.668733 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:53.668707    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/auth-test-model-ead650-kserve-d69f6f44c-pngn8"]
Jul 07 00:31:54.157873 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:54.157843    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3814ce87-bdf5-4367-80a7-5e750f05a1d3" path="/var/lib/kubelet/pods/3814ce87-bdf5-4367-80a7-5e750f05a1d3/volumes"
Jul 07 00:31:54.158240 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:54.158226    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cc1e299-35cc-40ab-83fd-454e7fb2c06f" path="/var/lib/kubelet/pods/5cc1e299-35cc-40ab-83fd-454e7fb2c06f/volumes"
Jul 07 00:31:58.458319 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:58.458274    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:58.458793 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:58.458339    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:58.470903 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:58.470877    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:31:59.382088 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:31:59.382057    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:32:10.354811 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.354733    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"]
Jul 07 00:32:10.355162 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.355101    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cc1e299-35cc-40ab-83fd-454e7fb2c06f" containerName="main"
Jul 07 00:32:10.355162 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.355112    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cc1e299-35cc-40ab-83fd-454e7fb2c06f" containerName="main"
Jul 07 00:32:10.355162 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.355133    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3814ce87-bdf5-4367-80a7-5e750f05a1d3" containerName="main"
Jul 07 00:32:10.355162 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.355138    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3814ce87-bdf5-4367-80a7-5e750f05a1d3" containerName="main"
Jul 07 00:32:10.355325 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.355202    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cc1e299-35cc-40ab-83fd-454e7fb2c06f" containerName="main"
Jul 07 00:32:10.355325 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.355210    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3814ce87-bdf5-4367-80a7-5e750f05a1d3" containerName="main"
Jul 07 00:32:10.360013 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.359987    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.363610 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.363585    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"default-dockercfg-6zdqv\""
Jul 07 00:32:10.363610 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.363596    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"openshift-service-ca.crt\""
Jul 07 00:32:10.363610 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.363586    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"sub-test-model-ead650-kserve-self-signed-certs\""
Jul 07 00:32:10.363830 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.363586    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"kube-root-ca.crt\""
Jul 07 00:32:10.367857 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.367829    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"]
Jul 07 00:32:10.419471 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.419438    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tls-certs\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.419471 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.419474    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-home\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.419727 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.419551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tmp-dir\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.419727 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.419585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-dshm\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.419727 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.419628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn4ll\" (UniqueName: \"kubernetes.io/projected/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-kube-api-access-jn4ll\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.419727 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.419687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-model-cache\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.520771 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.520731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tmp-dir\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.520771 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.520772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-dshm\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.520804    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn4ll\" (UniqueName: \"kubernetes.io/projected/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-kube-api-access-jn4ll\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.520867    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-model-cache\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.520917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tls-certs\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.520943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-home\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521312 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.521285    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tmp-dir\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521373 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.521331    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-model-cache\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.521373 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.521348    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-home\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.523358 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.523325    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-dshm\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.523576 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.523554    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tls-certs\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.531727 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.531702    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn4ll\" (UniqueName: \"kubernetes.io/projected/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-kube-api-access-jn4ll\") pod \"sub-test-model-ead650-kserve-748dfd7bd5-g2mvh\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") " pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.671635 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.671509    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:10.810654 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:10.810628    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"]
Jul 07 00:32:10.813290 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:32:10.813262    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d14d3ab_b341_4ffa_ac1d_d465ac2649de.slice/crio-7893ea431f3bb68f912f1db04f5cd58a0478dfa21abf2b764bb4901d8661c6a7 WatchSource:0}: Error finding container 7893ea431f3bb68f912f1db04f5cd58a0478dfa21abf2b764bb4901d8661c6a7: Status 404 returned error can't find the container with id 7893ea431f3bb68f912f1db04f5cd58a0478dfa21abf2b764bb4901d8661c6a7
Jul 07 00:32:11.429433 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:11.429394    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh" event={"ID":"0d14d3ab-b341-4ffa-ac1d-d465ac2649de","Type":"ContainerStarted","Data":"a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727"}
Jul 07 00:32:11.429841 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:11.429443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh" event={"ID":"0d14d3ab-b341-4ffa-ac1d-d465ac2649de","Type":"ContainerStarted","Data":"7893ea431f3bb68f912f1db04f5cd58a0478dfa21abf2b764bb4901d8661c6a7"}
Jul 07 00:32:11.446751 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:11.446698    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh" podStartSLOduration=1.446680255 podStartE2EDuration="1.446680255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:11.445358321 +0000 UTC m=+2155.810662093" watchObservedRunningTime="2026-07-07 00:32:11.446680255 +0000 UTC m=+2155.811984000"
Jul 07 00:32:20.672664 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:20.672611    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:20.673235 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:20.672686    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:20.685756 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:20.685726    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:21.490178 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:21.490139    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:32:57.073796 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:57.073763    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"]
Jul 07 00:32:58.030163 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:32:58.030126    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"]
Jul 07 00:33:03.460824 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.460790    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"]
Jul 07 00:33:03.464356 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.464334    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.466843 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.466814    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5f1a0\"/\"rate-test-model-f5f1a0-kserve-self-signed-certs\""
Jul 07 00:33:03.474786 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.474763    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"]
Jul 07 00:33:03.554740 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.554692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-dshm\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.554740 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.554748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4633140a-0a0b-4092-b199-9968eff1db7b-tls-certs\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.554987 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.554773    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-home\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.554987 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.554884    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-model-cache\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.554987 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.554953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-tmp-dir\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.554987 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.554971    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdfxq\" (UniqueName: \"kubernetes.io/projected/4633140a-0a0b-4092-b199-9968eff1db7b-kube-api-access-gdfxq\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656009 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.655961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-dshm\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656223 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656022    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4633140a-0a0b-4092-b199-9968eff1db7b-tls-certs\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656223 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-home\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656223 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-model-cache\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656223 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-tmp-dir\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656223 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdfxq\" (UniqueName: \"kubernetes.io/projected/4633140a-0a0b-4092-b199-9968eff1db7b-kube-api-access-gdfxq\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656501 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656477    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-home\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656599 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656564    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-model-cache\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.656644 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.656614    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-tmp-dir\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.658468 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.658449    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-dshm\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.658864 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.658848    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4633140a-0a0b-4092-b199-9968eff1db7b-tls-certs\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.665022 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.665001    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdfxq\" (UniqueName: \"kubernetes.io/projected/4633140a-0a0b-4092-b199-9968eff1db7b-kube-api-access-gdfxq\") pod \"rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") " pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.775296 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.775253    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:03.919769 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:03.919740    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"]
Jul 07 00:33:03.922053 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:33:03.922023    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4633140a_0a0b_4092_b199_9968eff1db7b.slice/crio-b50048a86b5fc7748846b6839f022f6597d2f48851dc988172a9af2c55d6c924 WatchSource:0}: Error finding container b50048a86b5fc7748846b6839f022f6597d2f48851dc988172a9af2c55d6c924: Status 404 returned error can't find the container with id b50048a86b5fc7748846b6839f022f6597d2f48851dc988172a9af2c55d6c924
Jul 07 00:33:04.683995 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:04.683958    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v" event={"ID":"4633140a-0a0b-4092-b199-9968eff1db7b","Type":"ContainerStarted","Data":"f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7"}
Jul 07 00:33:04.684484 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:04.684006    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v" event={"ID":"4633140a-0a0b-4092-b199-9968eff1db7b","Type":"ContainerStarted","Data":"b50048a86b5fc7748846b6839f022f6597d2f48851dc988172a9af2c55d6c924"}
Jul 07 00:33:04.702840 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:04.702785    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v" podStartSLOduration=1.7027679180000002 podStartE2EDuration="1.702767918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:33:04.700843887 +0000 UTC m=+2209.066147658" watchObservedRunningTime="2026-07-07 00:33:04.702767918 +0000 UTC m=+2209.068071663"
Jul 07 00:33:12.094962 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.094901    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7" podUID="80ae6f99-e84c-4be2-a468-31619158fe22" containerName="main" containerID="cri-o://5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201" gracePeriod=45
Jul 07 00:33:12.348820 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.348751    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:33:12.441452 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441420    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-model-cache\") pod \"80ae6f99-e84c-4be2-a468-31619158fe22\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") "
Jul 07 00:33:12.441676 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ae6f99-e84c-4be2-a468-31619158fe22-tls-certs\") pod \"80ae6f99-e84c-4be2-a468-31619158fe22\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") "
Jul 07 00:33:12.441676 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441510    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-home\") pod \"80ae6f99-e84c-4be2-a468-31619158fe22\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") "
Jul 07 00:33:12.441676 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-dshm\") pod \"80ae6f99-e84c-4be2-a468-31619158fe22\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") "
Jul 07 00:33:12.441676 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441604    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjqxq\" (UniqueName: \"kubernetes.io/projected/80ae6f99-e84c-4be2-a468-31619158fe22-kube-api-access-fjqxq\") pod \"80ae6f99-e84c-4be2-a468-31619158fe22\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") "
Jul 07 00:33:12.441676 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441656    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-tmp-dir\") pod \"80ae6f99-e84c-4be2-a468-31619158fe22\" (UID: \"80ae6f99-e84c-4be2-a468-31619158fe22\") "
Jul 07 00:33:12.441954 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441806    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-model-cache" (OuterVolumeSpecName: "model-cache") pod "80ae6f99-e84c-4be2-a468-31619158fe22" (UID: "80ae6f99-e84c-4be2-a468-31619158fe22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:12.441954 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441842    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-home" (OuterVolumeSpecName: "home") pod "80ae6f99-e84c-4be2-a468-31619158fe22" (UID: "80ae6f99-e84c-4be2-a468-31619158fe22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:12.441954 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.441930    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80ae6f99-e84c-4be2-a468-31619158fe22" (UID: "80ae6f99-e84c-4be2-a468-31619158fe22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:12.442121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.442048    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:12.442121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.442065    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:12.442121 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.442078    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:12.443940 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.443911    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-dshm" (OuterVolumeSpecName: "dshm") pod "80ae6f99-e84c-4be2-a468-31619158fe22" (UID: "80ae6f99-e84c-4be2-a468-31619158fe22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:12.444065 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.443957    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ae6f99-e84c-4be2-a468-31619158fe22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80ae6f99-e84c-4be2-a468-31619158fe22" (UID: "80ae6f99-e84c-4be2-a468-31619158fe22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:33:12.444065 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.444026    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ae6f99-e84c-4be2-a468-31619158fe22-kube-api-access-fjqxq" (OuterVolumeSpecName: "kube-api-access-fjqxq") pod "80ae6f99-e84c-4be2-a468-31619158fe22" (UID: "80ae6f99-e84c-4be2-a468-31619158fe22"). InnerVolumeSpecName "kube-api-access-fjqxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:33:12.543001 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.542961    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ae6f99-e84c-4be2-a468-31619158fe22-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:12.543001 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.542993    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ae6f99-e84c-4be2-a468-31619158fe22-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:12.543001 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.543003    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjqxq\" (UniqueName: \"kubernetes.io/projected/80ae6f99-e84c-4be2-a468-31619158fe22-kube-api-access-fjqxq\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:12.727318 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.727220    2583 generic.go:356] "Generic (PLEG): container finished" podID="80ae6f99-e84c-4be2-a468-31619158fe22" containerID="5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201" exitCode=0
Jul 07 00:33:12.727318 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.727276    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"
Jul 07 00:33:12.727542 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.727312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7" event={"ID":"80ae6f99-e84c-4be2-a468-31619158fe22","Type":"ContainerDied","Data":"5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201"}
Jul 07 00:33:12.727542 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.727370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7" event={"ID":"80ae6f99-e84c-4be2-a468-31619158fe22","Type":"ContainerDied","Data":"00f8278a5239773221e35f38c40d4285f13a1ba40f167a4b45c35bc401e36858"}
Jul 07 00:33:12.727542 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.727394    2583 scope.go:117] "RemoveContainer" containerID="5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201"
Jul 07 00:33:12.737882 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.737866    2583 scope.go:117] "RemoveContainer" containerID="5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201"
Jul 07 00:33:12.738167 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:33:12.738143    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201\": container with ID starting with 5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201 not found: ID does not exist" containerID="5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201"
Jul 07 00:33:12.738217 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.738189    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201"} err="failed to get container status \"5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201\": rpc error: code = NotFound desc = could not find container \"5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201\": container with ID starting with 5ec651c0d6b997675c774e198e60a2e21a21a6f9adce69df204b6d269237e201 not found: ID does not exist"
Jul 07 00:33:12.750829 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.750795    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"]
Jul 07 00:33:12.754926 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:12.754901    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/sub-test-model-f5f1a0-kserve-7f7bf47845-cpqs7"]
Jul 07 00:33:13.051332 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.051292    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh" podUID="0d14d3ab-b341-4ffa-ac1d-d465ac2649de" containerName="main" containerID="cri-o://a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727" gracePeriod=45
Jul 07 00:33:13.309312 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.309255    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:33:13.351216 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351185    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn4ll\" (UniqueName: \"kubernetes.io/projected/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-kube-api-access-jn4ll\") pod \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") "
Jul 07 00:33:13.351216 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351219    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-model-cache\") pod \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") "
Jul 07 00:33:13.351467 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351251    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tmp-dir\") pod \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") "
Jul 07 00:33:13.351467 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351383    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-dshm\") pod \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") "
Jul 07 00:33:13.351467 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351433    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-home\") pod \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") "
Jul 07 00:33:13.351658 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351477    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tls-certs\") pod \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\" (UID: \"0d14d3ab-b341-4ffa-ac1d-d465ac2649de\") "
Jul 07 00:33:13.351658 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351519    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d14d3ab-b341-4ffa-ac1d-d465ac2649de" (UID: "0d14d3ab-b341-4ffa-ac1d-d465ac2649de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:13.351658 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351563    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d14d3ab-b341-4ffa-ac1d-d465ac2649de" (UID: "0d14d3ab-b341-4ffa-ac1d-d465ac2649de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:13.351820 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351763    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-home" (OuterVolumeSpecName: "home") pod "0d14d3ab-b341-4ffa-ac1d-d465ac2649de" (UID: "0d14d3ab-b341-4ffa-ac1d-d465ac2649de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:13.351872 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351850    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:13.351872 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351866    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:13.352005 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.351880    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:13.353630 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.353599    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-dshm" (OuterVolumeSpecName: "dshm") pod "0d14d3ab-b341-4ffa-ac1d-d465ac2649de" (UID: "0d14d3ab-b341-4ffa-ac1d-d465ac2649de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:13.353719 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.353651    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d14d3ab-b341-4ffa-ac1d-d465ac2649de" (UID: "0d14d3ab-b341-4ffa-ac1d-d465ac2649de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:33:13.353719 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.353654    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-kube-api-access-jn4ll" (OuterVolumeSpecName: "kube-api-access-jn4ll") pod "0d14d3ab-b341-4ffa-ac1d-d465ac2649de" (UID: "0d14d3ab-b341-4ffa-ac1d-d465ac2649de"). InnerVolumeSpecName "kube-api-access-jn4ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:33:13.452633 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.452599    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:13.452633 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.452630    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn4ll\" (UniqueName: \"kubernetes.io/projected/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-kube-api-access-jn4ll\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:13.452633 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.452639    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d14d3ab-b341-4ffa-ac1d-d465ac2649de-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:33:13.734931 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.734897    2583 generic.go:356] "Generic (PLEG): container finished" podID="0d14d3ab-b341-4ffa-ac1d-d465ac2649de" containerID="a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727" exitCode=0
Jul 07 00:33:13.735075 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.734946    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"
Jul 07 00:33:13.735075 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.734945    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh" event={"ID":"0d14d3ab-b341-4ffa-ac1d-d465ac2649de","Type":"ContainerDied","Data":"a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727"}
Jul 07 00:33:13.735145 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.735071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh" event={"ID":"0d14d3ab-b341-4ffa-ac1d-d465ac2649de","Type":"ContainerDied","Data":"7893ea431f3bb68f912f1db04f5cd58a0478dfa21abf2b764bb4901d8661c6a7"}
Jul 07 00:33:13.735145 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.735096    2583 scope.go:117] "RemoveContainer" containerID="a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727"
Jul 07 00:33:13.744994 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.744975    2583 scope.go:117] "RemoveContainer" containerID="a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727"
Jul 07 00:33:13.745252 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:33:13.745233    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727\": container with ID starting with a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727 not found: ID does not exist" containerID="a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727"
Jul 07 00:33:13.745317 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.745258    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727"} err="failed to get container status \"a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727\": rpc error: code = NotFound desc = could not find container \"a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727\": container with ID starting with a68c9a0e7c0dcc90be36c4380d7a3b7879ef21a7a7d2c0303cad3c8717730727 not found: ID does not exist"
Jul 07 00:33:13.764719 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.764694    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"]
Jul 07 00:33:13.776080 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.776052    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/sub-test-model-ead650-kserve-748dfd7bd5-g2mvh"]
Jul 07 00:33:13.776176 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.776106    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:13.776176 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.776126    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:13.788938 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:13.788918    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:14.157936 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:14.157846    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d14d3ab-b341-4ffa-ac1d-d465ac2649de" path="/var/lib/kubelet/pods/0d14d3ab-b341-4ffa-ac1d-d465ac2649de/volumes"
Jul 07 00:33:14.158384 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:14.158363    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ae6f99-e84c-4be2-a468-31619158fe22" path="/var/lib/kubelet/pods/80ae6f99-e84c-4be2-a468-31619158fe22/volumes"
Jul 07 00:33:14.752066 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:14.752030    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:33:26.649314 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.649272    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"]
Jul 07 00:33:26.649885 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.649864    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ae6f99-e84c-4be2-a468-31619158fe22" containerName="main"
Jul 07 00:33:26.649979 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.649887    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ae6f99-e84c-4be2-a468-31619158fe22" containerName="main"
Jul 07 00:33:26.649979 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.649916    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d14d3ab-b341-4ffa-ac1d-d465ac2649de" containerName="main"
Jul 07 00:33:26.649979 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.649925    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d14d3ab-b341-4ffa-ac1d-d465ac2649de" containerName="main"
Jul 07 00:33:26.650125 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.650024    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ae6f99-e84c-4be2-a468-31619158fe22" containerName="main"
Jul 07 00:33:26.650125 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.650040    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d14d3ab-b341-4ffa-ac1d-d465ac2649de" containerName="main"
Jul 07 00:33:26.653208 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.653186    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.657101 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.657080    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"rate-test-model-ead650-kserve-self-signed-certs\""
Jul 07 00:33:26.657231 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.657080    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"kube-root-ca.crt\""
Jul 07 00:33:26.657231 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.657080    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"default-dockercfg-6zdqv\""
Jul 07 00:33:26.657231 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.657154    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ead650\"/\"openshift-service-ca.crt\""
Jul 07 00:33:26.663164 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.663140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"]
Jul 07 00:33:26.780212 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.780172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tls-certs\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.780396 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.780241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-home\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.780396 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.780292    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-dshm\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.780396 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.780334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn8zl\" (UniqueName: \"kubernetes.io/projected/a20e771a-d219-4b7c-b5b2-1065bd560dbf-kube-api-access-rn8zl\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.780396 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.780363    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tmp-dir\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.780568 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.780462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-model-cache\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.881877 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.881836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn8zl\" (UniqueName: \"kubernetes.io/projected/a20e771a-d219-4b7c-b5b2-1065bd560dbf-kube-api-access-rn8zl\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.881877 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.881885    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tmp-dir\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.881912    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-model-cache\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.881956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tls-certs\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.882014    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-home\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882126 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.882040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-dshm\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882367 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.882340    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tmp-dir\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882421 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.882367    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-model-cache\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.882471 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.882449    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-home\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.884623 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.884593    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-dshm\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.884854 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.884835    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tls-certs\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.891485 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.891463    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn8zl\" (UniqueName: \"kubernetes.io/projected/a20e771a-d219-4b7c-b5b2-1065bd560dbf-kube-api-access-rn8zl\") pod \"rate-test-model-ead650-kserve-7584cf6667-j8kxw\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") " pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:26.963234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:26.963132    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:27.108899 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:27.108871    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"]
Jul 07 00:33:27.111244 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:33:27.111198    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda20e771a_d219_4b7c_b5b2_1065bd560dbf.slice/crio-48c5cc8e59e63ca463a2a9ddddf42ecf20a2aaf955b0bb7d9393defa83cdd232 WatchSource:0}: Error finding container 48c5cc8e59e63ca463a2a9ddddf42ecf20a2aaf955b0bb7d9393defa83cdd232: Status 404 returned error can't find the container with id 48c5cc8e59e63ca463a2a9ddddf42ecf20a2aaf955b0bb7d9393defa83cdd232
Jul 07 00:33:27.804719 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:27.804683    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw" event={"ID":"a20e771a-d219-4b7c-b5b2-1065bd560dbf","Type":"ContainerStarted","Data":"ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f"}
Jul 07 00:33:27.804719 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:27.804727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw" event={"ID":"a20e771a-d219-4b7c-b5b2-1065bd560dbf","Type":"ContainerStarted","Data":"48c5cc8e59e63ca463a2a9ddddf42ecf20a2aaf955b0bb7d9393defa83cdd232"}
Jul 07 00:33:27.822517 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:27.822461    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw" podStartSLOduration=1.8224442939999999 podStartE2EDuration="1.822444294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:33:27.820809295 +0000 UTC m=+2232.186113064" watchObservedRunningTime="2026-07-07 00:33:27.822444294 +0000 UTC m=+2232.187748039"
Jul 07 00:33:36.963846 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:36.963764    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:36.963846 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:36.963820    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:36.976404 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:36.976377    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:33:37.869192 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:33:37.869161    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:34:08.677064 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:08.677029    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"]
Jul 07 00:34:09.573658 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:09.573620    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"]
Jul 07 00:34:23.697961 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:23.697908    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v" podUID="4633140a-0a0b-4092-b199-9968eff1db7b" containerName="main" containerID="cri-o://f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7" gracePeriod=45
Jul 07 00:34:23.951147 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:23.951077    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:34:24.043337 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043289    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-model-cache\") pod \"4633140a-0a0b-4092-b199-9968eff1db7b\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") "
Jul 07 00:34:24.043573 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043359    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-dshm\") pod \"4633140a-0a0b-4092-b199-9968eff1db7b\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") "
Jul 07 00:34:24.043573 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043421    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdfxq\" (UniqueName: \"kubernetes.io/projected/4633140a-0a0b-4092-b199-9968eff1db7b-kube-api-access-gdfxq\") pod \"4633140a-0a0b-4092-b199-9968eff1db7b\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") "
Jul 07 00:34:24.043573 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043458    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4633140a-0a0b-4092-b199-9968eff1db7b-tls-certs\") pod \"4633140a-0a0b-4092-b199-9968eff1db7b\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") "
Jul 07 00:34:24.043573 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043487    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-home\") pod \"4633140a-0a0b-4092-b199-9968eff1db7b\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") "
Jul 07 00:34:24.043573 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043507    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-tmp-dir\") pod \"4633140a-0a0b-4092-b199-9968eff1db7b\" (UID: \"4633140a-0a0b-4092-b199-9968eff1db7b\") "
Jul 07 00:34:24.043849 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043610    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4633140a-0a0b-4092-b199-9968eff1db7b" (UID: "4633140a-0a0b-4092-b199-9968eff1db7b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.043849 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043763    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-home" (OuterVolumeSpecName: "home") pod "4633140a-0a0b-4092-b199-9968eff1db7b" (UID: "4633140a-0a0b-4092-b199-9968eff1db7b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.043849 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043828    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4633140a-0a0b-4092-b199-9968eff1db7b" (UID: "4633140a-0a0b-4092-b199-9968eff1db7b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.043961 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043945    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.043996 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043963    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.044047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.043994    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.045790 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.045765    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-dshm" (OuterVolumeSpecName: "dshm") pod "4633140a-0a0b-4092-b199-9968eff1db7b" (UID: "4633140a-0a0b-4092-b199-9968eff1db7b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.045790 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.045764    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4633140a-0a0b-4092-b199-9968eff1db7b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4633140a-0a0b-4092-b199-9968eff1db7b" (UID: "4633140a-0a0b-4092-b199-9968eff1db7b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:34:24.045964 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.045814    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4633140a-0a0b-4092-b199-9968eff1db7b-kube-api-access-gdfxq" (OuterVolumeSpecName: "kube-api-access-gdfxq") pod "4633140a-0a0b-4092-b199-9968eff1db7b" (UID: "4633140a-0a0b-4092-b199-9968eff1db7b"). InnerVolumeSpecName "kube-api-access-gdfxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:34:24.080491 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.080454    2583 generic.go:356] "Generic (PLEG): container finished" podID="4633140a-0a0b-4092-b199-9968eff1db7b" containerID="f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7" exitCode=0
Jul 07 00:34:24.080675 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.080504    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"
Jul 07 00:34:24.080675 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.080568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v" event={"ID":"4633140a-0a0b-4092-b199-9968eff1db7b","Type":"ContainerDied","Data":"f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7"}
Jul 07 00:34:24.080675 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.080620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v" event={"ID":"4633140a-0a0b-4092-b199-9968eff1db7b","Type":"ContainerDied","Data":"b50048a86b5fc7748846b6839f022f6597d2f48851dc988172a9af2c55d6c924"}
Jul 07 00:34:24.080675 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.080644    2583 scope.go:117] "RemoveContainer" containerID="f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7"
Jul 07 00:34:24.092729 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.092712    2583 scope.go:117] "RemoveContainer" containerID="f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7"
Jul 07 00:34:24.092993 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:34:24.092975    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7\": container with ID starting with f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7 not found: ID does not exist" containerID="f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7"
Jul 07 00:34:24.093039 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.093003    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7"} err="failed to get container status \"f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7\": rpc error: code = NotFound desc = could not find container \"f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7\": container with ID starting with f4b332b859d56fa552e3009e8bf71deed560547da45caab3e3a185f16ff0d3f7 not found: ID does not exist"
Jul 07 00:34:24.104950 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.104918    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"]
Jul 07 00:34:24.109089 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.109068    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5f1a0/rate-test-model-f5f1a0-kserve-745f87c9b7-t4x7v"]
Jul 07 00:34:24.145113 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.145085    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4633140a-0a0b-4092-b199-9968eff1db7b-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.145113 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.145112    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdfxq\" (UniqueName: \"kubernetes.io/projected/4633140a-0a0b-4092-b199-9968eff1db7b-kube-api-access-gdfxq\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.145247 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.145121    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4633140a-0a0b-4092-b199-9968eff1db7b-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.157203 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.157180    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4633140a-0a0b-4092-b199-9968eff1db7b" path="/var/lib/kubelet/pods/4633140a-0a0b-4092-b199-9968eff1db7b/volumes"
Jul 07 00:34:24.594486 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.594437    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw" podUID="a20e771a-d219-4b7c-b5b2-1065bd560dbf" containerName="main" containerID="cri-o://ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f" gracePeriod=45
Jul 07 00:34:24.852219 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.852142    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:34:24.953252 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953217    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-model-cache\") pod \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") "
Jul 07 00:34:24.953252 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953253    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tmp-dir\") pod \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") "
Jul 07 00:34:24.953510 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953329    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn8zl\" (UniqueName: \"kubernetes.io/projected/a20e771a-d219-4b7c-b5b2-1065bd560dbf-kube-api-access-rn8zl\") pod \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") "
Jul 07 00:34:24.953510 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953367    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-home\") pod \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") "
Jul 07 00:34:24.953510 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953382    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tls-certs\") pod \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") "
Jul 07 00:34:24.953510 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953397    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-dshm\") pod \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\" (UID: \"a20e771a-d219-4b7c-b5b2-1065bd560dbf\") "
Jul 07 00:34:24.953510 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953483    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-model-cache" (OuterVolumeSpecName: "model-cache") pod "a20e771a-d219-4b7c-b5b2-1065bd560dbf" (UID: "a20e771a-d219-4b7c-b5b2-1065bd560dbf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.953767 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953611    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a20e771a-d219-4b7c-b5b2-1065bd560dbf" (UID: "a20e771a-d219-4b7c-b5b2-1065bd560dbf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.953767 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953655    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-home" (OuterVolumeSpecName: "home") pod "a20e771a-d219-4b7c-b5b2-1065bd560dbf" (UID: "a20e771a-d219-4b7c-b5b2-1065bd560dbf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:24.953767 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953760    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-home\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.953865 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953773    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-model-cache\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.953865 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.953792    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tmp-dir\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:24.955741 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.955707    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a20e771a-d219-4b7c-b5b2-1065bd560dbf-kube-api-access-rn8zl" (OuterVolumeSpecName: "kube-api-access-rn8zl") pod "a20e771a-d219-4b7c-b5b2-1065bd560dbf" (UID: "a20e771a-d219-4b7c-b5b2-1065bd560dbf"). InnerVolumeSpecName "kube-api-access-rn8zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:34:24.955894 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.955764    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a20e771a-d219-4b7c-b5b2-1065bd560dbf" (UID: "a20e771a-d219-4b7c-b5b2-1065bd560dbf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:34:24.955894 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:24.955794    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-dshm" (OuterVolumeSpecName: "dshm") pod "a20e771a-d219-4b7c-b5b2-1065bd560dbf" (UID: "a20e771a-d219-4b7c-b5b2-1065bd560dbf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:34:25.054643 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.054609    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn8zl\" (UniqueName: \"kubernetes.io/projected/a20e771a-d219-4b7c-b5b2-1065bd560dbf-kube-api-access-rn8zl\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:25.054643 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.054635    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20e771a-d219-4b7c-b5b2-1065bd560dbf-tls-certs\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:25.054643 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.054647    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20e771a-d219-4b7c-b5b2-1065bd560dbf-dshm\") on node \"ip-10-0-141-68.ec2.internal\" DevicePath \"\""
Jul 07 00:34:25.086501 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.086467    2583 generic.go:356] "Generic (PLEG): container finished" podID="a20e771a-d219-4b7c-b5b2-1065bd560dbf" containerID="ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f" exitCode=0
Jul 07 00:34:25.086691 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.086511    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"
Jul 07 00:34:25.086691 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.086557    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw" event={"ID":"a20e771a-d219-4b7c-b5b2-1065bd560dbf","Type":"ContainerDied","Data":"ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f"}
Jul 07 00:34:25.086691 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.086602    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw" event={"ID":"a20e771a-d219-4b7c-b5b2-1065bd560dbf","Type":"ContainerDied","Data":"48c5cc8e59e63ca463a2a9ddddf42ecf20a2aaf955b0bb7d9393defa83cdd232"}
Jul 07 00:34:25.086691 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.086627    2583 scope.go:117] "RemoveContainer" containerID="ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f"
Jul 07 00:34:25.095907 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.095890    2583 scope.go:117] "RemoveContainer" containerID="ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f"
Jul 07 00:34:25.096193 ip-10-0-141-68 kubenswrapper[2583]: E0707 00:34:25.096173    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f\": container with ID starting with ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f not found: ID does not exist" containerID="ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f"
Jul 07 00:34:25.096244 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.096202    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f"} err="failed to get container status \"ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f\": rpc error: code = NotFound desc = could not find container \"ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f\": container with ID starting with ff8081a0bbcf0ea4b9a77c312f0a801a538dced219c013c4cf735d8a7043149f not found: ID does not exist"
Jul 07 00:34:25.108211 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.108150    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"]
Jul 07 00:34:25.115775 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:25.114868    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ead650/rate-test-model-ead650-kserve-7584cf6667-j8kxw"]
Jul 07 00:34:26.157206 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:34:26.157160    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a20e771a-d219-4b7c-b5b2-1065bd560dbf" path="/var/lib/kubelet/pods/a20e771a-d219-4b7c-b5b2-1065bd560dbf/volumes"
Jul 07 00:35:06.790209 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:06.790116    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d7997cf4b-sdvxl_9cbfe34d-8232-4c98-82ae-f09c99b79056/copy-manifests/0.log"
Jul 07 00:35:06.799208 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:06.799179    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d7997cf4b-sdvxl_9cbfe34d-8232-4c98-82ae-f09c99b79056/manager/0.log"
Jul 07 00:35:06.911686 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:06.911655    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rrqrs_b7312506-02ca-4a85-a7d1-7cf7f169d17b/postgres/0.log"
Jul 07 00:35:08.496045 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:08.496018    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tfkrq_e7bf67cb-f796-47dc-ba58-18f1d163786b/kuadrant-console-plugin/0.log"
Jul 07 00:35:09.544601 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:09.544564    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7697f65cb7-fd4qc_c6f74e76-027a-41d9-a240-6e7248af829d/kube-auth-proxy/0.log"
Jul 07 00:35:10.002342 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:10.002311    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9mpf7_cf168186-0bb9-44ca-aa36-1d4b3b11f1c8/payload-processing/0.log"
Jul 07 00:35:10.455251 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:10.455160    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-n6tqr_7f34f6ad-7e13-4015-ada0-f27d7d44e534/main/0.log"
Jul 07 00:35:10.672512 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:10.672478    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-tdwk4_bd29d5c8-1c55-40ff-9708-fdabbe65327f/main/0.log"
Jul 07 00:35:11.019725 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:11.019690    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7hzmx_d79b616e-ed43-4f8e-b0b4-8a46ebf46ff1/main/0.log"
Jul 07 00:35:16.316909 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:16.316875    2583 scope.go:117] "RemoveContainer" containerID="b529137f50d6de52c12c282e06a7ba1b3ff72a385e2c6d3c5e23e5730d8c9459"
Jul 07 00:35:18.453379 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:18.453346    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zkcfs_da53e071-0353-4538-8837-240274613bc2/global-pull-secret-syncer/0.log"
Jul 07 00:35:18.584802 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:18.584761    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wqkxt_8a46aeb8-8183-4cf5-aa3e-c84bcc975b5f/konnectivity-agent/0.log"
Jul 07 00:35:18.653865 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:18.653836    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-68.ec2.internal_a09046d52a93c64b9a467e152c6a69f0/haproxy/0.log"
Jul 07 00:35:24.002365 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:24.002335    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tfkrq_e7bf67cb-f796-47dc-ba58-18f1d163786b/kuadrant-console-plugin/0.log"
Jul 07 00:35:26.056881 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.056846    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/alertmanager/0.log"
Jul 07 00:35:26.082345 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.082314    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/config-reloader/0.log"
Jul 07 00:35:26.107842 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.107816    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/kube-rbac-proxy-web/0.log"
Jul 07 00:35:26.132031 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.132002    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/kube-rbac-proxy/0.log"
Jul 07 00:35:26.161069 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.157958    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/kube-rbac-proxy-metric/0.log"
Jul 07 00:35:26.185583 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.185547    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/prom-label-proxy/0.log"
Jul 07 00:35:26.212155 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.212109    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fcf97620-1bc7-4916-ae42-f79d6f3d36c6/init-config-reloader/0.log"
Jul 07 00:35:26.291219 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.291177    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vrbgs_39ec36b4-f2f4-4895-bf89-c7f688d56b14/kube-state-metrics/0.log"
Jul 07 00:35:26.313125 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.313029    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vrbgs_39ec36b4-f2f4-4895-bf89-c7f688d56b14/kube-rbac-proxy-main/0.log"
Jul 07 00:35:26.335843 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.335809    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vrbgs_39ec36b4-f2f4-4895-bf89-c7f688d56b14/kube-rbac-proxy-self/0.log"
Jul 07 00:35:26.401067 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.401017    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-c5x2n_6bced467-26ca-441e-b1bd-fe12027bf6be/monitoring-plugin/0.log"
Jul 07 00:35:26.588432 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.588329    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dmkxm_2faf3977-df0c-4d32-af9e-1a25cd8a3996/node-exporter/0.log"
Jul 07 00:35:26.608845 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.608810    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dmkxm_2faf3977-df0c-4d32-af9e-1a25cd8a3996/kube-rbac-proxy/0.log"
Jul 07 00:35:26.633685 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.633670    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dmkxm_2faf3977-df0c-4d32-af9e-1a25cd8a3996/init-textfile/0.log"
Jul 07 00:35:26.744630 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.744591    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/prometheus/0.log"
Jul 07 00:35:26.765047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.765013    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/config-reloader/0.log"
Jul 07 00:35:26.786044 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.786011    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/thanos-sidecar/0.log"
Jul 07 00:35:26.808904 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.808873    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/kube-rbac-proxy-web/0.log"
Jul 07 00:35:26.830918 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.830890    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/kube-rbac-proxy/0.log"
Jul 07 00:35:26.853826 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.853739    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/kube-rbac-proxy-thanos/0.log"
Jul 07 00:35:26.880007 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.879975    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7a11914c-6465-455b-98cd-34522022977e/init-config-reloader/0.log"
Jul 07 00:35:26.913800 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.913769    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q98k7_f963865e-130e-4462-a2fc-060e814c82c5/prometheus-operator/0.log"
Jul 07 00:35:26.933707 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.933674    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q98k7_f963865e-130e-4462-a2fc-060e814c82c5/kube-rbac-proxy/0.log"
Jul 07 00:35:26.958864 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.958828    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-jg8rv_bdb4975e-e6af-4890-add8-22ef81d14bf6/prometheus-operator-admission-webhook/0.log"
Jul 07 00:35:26.988984 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:26.988950    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68fff647f-jw9jx_5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee/telemeter-client/0.log"
Jul 07 00:35:27.013591 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:27.013507    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68fff647f-jw9jx_5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee/reload/0.log"
Jul 07 00:35:27.034634 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:27.034603    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68fff647f-jw9jx_5ffaf726-bc0a-4db6-bd89-b2c4fffc32ee/kube-rbac-proxy/0.log"
Jul 07 00:35:28.269143 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269110    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"]
Jul 07 00:35:28.269815 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269785    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a20e771a-d219-4b7c-b5b2-1065bd560dbf" containerName="main"
Jul 07 00:35:28.269815 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269812    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a20e771a-d219-4b7c-b5b2-1065bd560dbf" containerName="main"
Jul 07 00:35:28.270032 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269857    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4633140a-0a0b-4092-b199-9968eff1db7b" containerName="main"
Jul 07 00:35:28.270032 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269866    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4633140a-0a0b-4092-b199-9968eff1db7b" containerName="main"
Jul 07 00:35:28.270032 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269964    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4633140a-0a0b-4092-b199-9968eff1db7b" containerName="main"
Jul 07 00:35:28.270032 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.269976    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a20e771a-d219-4b7c-b5b2-1065bd560dbf" containerName="main"
Jul 07 00:35:28.273248 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.273227    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.275917 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.275893    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6khfq\"/\"default-dockercfg-hjcwq\""
Jul 07 00:35:28.276033 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.275920    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6khfq\"/\"kube-root-ca.crt\""
Jul 07 00:35:28.276796 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.276782    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6khfq\"/\"openshift-service-ca.crt\""
Jul 07 00:35:28.283970 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.283840    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"]
Jul 07 00:35:28.346309 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.346269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-sys\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.346520 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.346318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69csc\" (UniqueName: \"kubernetes.io/projected/0e8fe8c0-33f9-4b97-902f-0d2472486c27-kube-api-access-69csc\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.346520 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.346341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-lib-modules\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.346520 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.346424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-proc\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.346520 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.346458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-podres\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.447776 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447661    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-proc\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.447776 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447699    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-podres\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.447776 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-sys\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.447776 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69csc\" (UniqueName: \"kubernetes.io/projected/0e8fe8c0-33f9-4b97-902f-0d2472486c27-kube-api-access-69csc\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.456234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-lib-modules\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.456234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447805    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-proc\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.456234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447906    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-sys\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.456234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447930    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-lib-modules\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.456234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.447930    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0e8fe8c0-33f9-4b97-902f-0d2472486c27-podres\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.456234 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.455375    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69csc\" (UniqueName: \"kubernetes.io/projected/0e8fe8c0-33f9-4b97-902f-0d2472486c27-kube-api-access-69csc\") pod \"perf-node-gather-daemonset-tnrqt\" (UID: \"0e8fe8c0-33f9-4b97-902f-0d2472486c27\") " pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.585766 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.585654    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:28.726220 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:28.726155    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"]
Jul 07 00:35:28.728816 ip-10-0-141-68 kubenswrapper[2583]: W0707 00:35:28.728786    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0e8fe8c0_33f9_4b97_902f_0d2472486c27.slice/crio-b9281ada31ae01eb3d404b09920febd2a7b4a0d7006f0730f551e9299fdcffbe WatchSource:0}: Error finding container b9281ada31ae01eb3d404b09920febd2a7b4a0d7006f0730f551e9299fdcffbe: Status 404 returned error can't find the container with id b9281ada31ae01eb3d404b09920febd2a7b4a0d7006f0730f551e9299fdcffbe
Jul 07 00:35:29.396968 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:29.396928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt" event={"ID":"0e8fe8c0-33f9-4b97-902f-0d2472486c27","Type":"ContainerStarted","Data":"0068a5e3a9366a008e5a279df0a74bd373a779074c0facd1f41f1c78f27304ca"}
Jul 07 00:35:29.397346 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:29.396979    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt" event={"ID":"0e8fe8c0-33f9-4b97-902f-0d2472486c27","Type":"ContainerStarted","Data":"b9281ada31ae01eb3d404b09920febd2a7b4a0d7006f0730f551e9299fdcffbe"}
Jul 07 00:35:29.397346 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:29.397059    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:29.414203 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:29.414131    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt" podStartSLOduration=1.41411504 podStartE2EDuration="1.41411504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:35:29.412580372 +0000 UTC m=+2353.777884143" watchObservedRunningTime="2026-07-07 00:35:29.41411504 +0000 UTC m=+2353.779418785"
Jul 07 00:35:29.577460 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:29.577433    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-t2kph_e00e698a-1f4f-410b-9897-39b873210865/download-server/0.log"
Jul 07 00:35:31.335047 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:31.334981    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvdb7_6896410a-be62-4c56-a0ef-13dca1e18d5c/dns/0.log"
Jul 07 00:35:31.358446 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:31.358408    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvdb7_6896410a-be62-4c56-a0ef-13dca1e18d5c/kube-rbac-proxy/0.log"
Jul 07 00:35:31.478681 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:31.478649    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xr9cd_bae566a7-243e-46e2-9bd7-e58e689eb474/dns-node-resolver/0.log"
Jul 07 00:35:32.084415 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:32.084373    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64f66b6665-lpmjm_b7ac5bca-caa4-4fbc-bf90-de9897830455/registry/0.log"
Jul 07 00:35:32.132944 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:32.132908    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w272k_70b6c117-9122-43d1-aa4f-bee759d72e46/node-ca/0.log"
Jul 07 00:35:33.356412 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:33.356379    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7697f65cb7-fd4qc_c6f74e76-027a-41d9-a240-6e7248af829d/kube-auth-proxy/0.log"
Jul 07 00:35:33.598788 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:33.598746    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9mpf7_cf168186-0bb9-44ca-aa36-1d4b3b11f1c8/payload-processing/0.log"
Jul 07 00:35:34.342302 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:34.342268    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q4mbb_9046d0c8-a0e1-4641-bbbd-b7f10d639dcb/serve-healthcheck-canary/0.log"
Jul 07 00:35:34.942185 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:34.942140    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mjk9s_93cdaebe-f266-44cb-b2f4-c2d3c9081cdb/kube-rbac-proxy/0.log"
Jul 07 00:35:34.963455 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:34.963423    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mjk9s_93cdaebe-f266-44cb-b2f4-c2d3c9081cdb/exporter/0.log"
Jul 07 00:35:34.986283 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:34.986242    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mjk9s_93cdaebe-f266-44cb-b2f4-c2d3c9081cdb/extractor/0.log"
Jul 07 00:35:35.413061 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:35.413035    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6khfq/perf-node-gather-daemonset-tnrqt"
Jul 07 00:35:38.091715 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:38.091684    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d7997cf4b-sdvxl_9cbfe34d-8232-4c98-82ae-f09c99b79056/manager/0.log"
Jul 07 00:35:38.113053 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:38.113020    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d7997cf4b-sdvxl_9cbfe34d-8232-4c98-82ae-f09c99b79056/copy-manifests/0.log"
Jul 07 00:35:38.140103 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:38.140077    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rrqrs_b7312506-02ca-4a85-a7d1-7cf7f169d17b/postgres/0.log"
Jul 07 00:35:39.461351 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:39.461316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59b45dbd94-bvht7_f0feca0b-94dc-4c1b-add4-20f18f358220/manager/0.log"
Jul 07 00:35:39.561453 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:39.561361    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7697f65cb7-fd4qc_c6f74e76-027a-41d9-a240-6e7248af829d/kube-auth-proxy/0.log"
Jul 07 00:35:39.811701 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:39.811608    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9mpf7_cf168186-0bb9-44ca-aa36-1d4b3b11f1c8/payload-processing/0.log"
Jul 07 00:35:41.305247 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:41.305216    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7697f65cb7-fd4qc_c6f74e76-027a-41d9-a240-6e7248af829d/kube-auth-proxy/0.log"
Jul 07 00:35:41.507650 ip-10-0-141-68 kubenswrapper[2583]: I0707 00:35:41.507599    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9mpf7_cf168186-0bb9-44ca-aa36-1d4b3b11f1c8/payload-processing/0.log"