Jul 22 15:11:07.392796 ip-10-0-143-231 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 15:11:07.392808 ip-10-0-143-231 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 15:11:07.392815 ip-10-0-143-231 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 15:11:07.393109 ip-10-0-143-231 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 15:11:17.642049 ip-10-0-143-231 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 15:11:17.642065 ip-10-0-143-231 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2e943d8ffb454628b8091522a9542977 --
Jul 22 15:13:10.447844 ip-10-0-143-231 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 15:13:10.814540 ip-10-0-143-231 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 15:13:10.814540 ip-10-0-143-231 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 15:13:10.814540 ip-10-0-143-231 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 15:13:10.814540 ip-10-0-143-231 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 15:13:10.814540 ip-10-0-143-231 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 15:13:10.815264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.814595    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 15:13:10.819933 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819913    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:13:10.819933 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819933    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819938    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819941    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819944    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819946    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819949    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819952    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819955    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819957    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819960    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819963    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819965    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819968    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819972    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819975    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819977    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819980    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819984    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819989    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:13:10.820007 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819993    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819996    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.819999    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820002    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820005    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820008    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820011    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820014    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820017    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820020    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820023    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820026    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820028    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820032    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820036    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820039    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820042    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820045    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820048    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820051    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:13:10.820465 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820053    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820057    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820059    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820062    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820064    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820067    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820069    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820072    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820075    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820078    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820080    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820082    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820085    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820088    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820091    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820094    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820096    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820099    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820101    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820104    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:13:10.820968 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820107    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820109    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820112    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820114    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820117    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820119    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820122    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820124    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820142    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820145    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820148    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820151    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820154    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820157    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820159    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820162    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820167    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820170    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820172    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820175    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:13:10.821450 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820179    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820183    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820185    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820188    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820191    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820195    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820198    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820201    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820203    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820206    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820209    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820211    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820214    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820217    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820219    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820222    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820224    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820227    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820229    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820962    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:13:10.821955 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820969    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820972    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820976    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820979    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820983    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820986    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820989    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820991    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820994    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820996    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.820999    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821002    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821004    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821007    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821009    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821012    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821014    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821017    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821020    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821023    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:13:10.822445 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821026    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821028    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821031    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821034    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821036    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821039    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821042    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821045    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821047    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821050    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821052    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821054    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821057    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821060    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821062    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821064    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821068    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821076    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821079    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821081    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:13:10.822959 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821084    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821086    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821089    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821091    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821094    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821097    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821099    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821102    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821104    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821107    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821111    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821114    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821117    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821119    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821122    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821125    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821127    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821130    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821132    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:13:10.823441 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821135    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821137    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821140    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821143    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821145    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821147    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821150    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821155    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821158    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821160    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821163    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821165    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821168    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821170    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821173    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821175    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821178    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821180    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821183    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821185    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:13:10.823919 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821187    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821190    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821194    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821196    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821199    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821202    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821205    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821207    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821211    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821214    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821217    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821219    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821222    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821226    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821229    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821232    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821236    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821240    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821243    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:13:10.824426 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821661    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821667    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821670    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821673    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821677    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821680    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821682    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821685    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821688    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821691    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821694    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821698    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821702    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821704    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821707    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821710    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821713    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821716    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821718    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821721    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:13:10.824899 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821724    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821726    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821729    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821731    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821734    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821737    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821740    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821742    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821745    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821748    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821751    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821753    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821756    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821758    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821761    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821763    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821766    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821769    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821771    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:13:10.825383 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821773    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821776    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821778    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821781    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821783    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821785    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821788    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821791    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821796    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821799    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821802    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821805    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821808    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821811    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821814    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821816    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821819    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821821    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821824    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:13:10.825889 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821827    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821830    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821832    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821835    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821838    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821841    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821843    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821846    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821849    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821851    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821854    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821856    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821859    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821862    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821864    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821867    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821869    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821872    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821874    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821878    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:13:10.826357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821880    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821885    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821888    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821891    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821894    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821896    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821898    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821901    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821903    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821906    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821911    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821914    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821916    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821919    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821921    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821924    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821927    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821929    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821932    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821934    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:13:10.826857 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.821937    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822014    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822022    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822030    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822037    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822045    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822049    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822055    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822062    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822065    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822068    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822072    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822075    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822080    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822083    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822086    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822089    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822092    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822095    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822099    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822102    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822105    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822109    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822112    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 15:13:10.827378 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822116    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822119    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822122    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822126    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822129    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822131    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822134    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822137    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822140    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822144    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822147    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822150    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822153    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822156    2580 flags.go:64] FLAG: --enable-server="true"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822160    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822165    2580 flags.go:64] FLAG: --event-burst="100"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822168    2580 flags.go:64] FLAG: --event-qps="50"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822170    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822173    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822176    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822180    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822185    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822188    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822191    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822194    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 15:13:10.827962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822196    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822199    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822202    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822205    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822208    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822211    2580 flags.go:64] FLAG: --feature-gates=""
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822215    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822218    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822221    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822224    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822227    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822230    2580 flags.go:64] FLAG: --help="false"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822233    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-231.ec2.internal"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822236    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822247    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822250    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822254    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822257    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822261    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822264    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822267    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822270    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822274    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822277    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 15:13:10.828592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822280    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822283    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822286    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822289    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822293    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822297    2580 flags.go:64] FLAG: --lock-file=""
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822299    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822302    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822305    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822311    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822314    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822317    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822319    2580 flags.go:64] FLAG: --logging-format="text"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822323    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822327    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822330    2580 flags.go:64] FLAG: --manifest-url=""
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822333    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822337    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822340    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822345    2580 flags.go:64] FLAG: --max-pods="110"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822348    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822351    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822353    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822356    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822359    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 15:13:10.829184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822362    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822365    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822372    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822375    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822379    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822382    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822385    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822391    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822394    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822397    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822400    2580 flags.go:64] FLAG: --port="10250"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822403    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822407    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a2078118e2c96539"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822410    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822412    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822415    2580 flags.go:64] FLAG: --register-node="true"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822418    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822422    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822425    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822428    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822430    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822435    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822437    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822441    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822444    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 15:13:10.829809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822446    2580 flags.go:64] FLAG: --runonce="false"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822449    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822452    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822456    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822459    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822462    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822465    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822468    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822471    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822474    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822477    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822480    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822483    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822499    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822503    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822506    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822512    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822514    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822517    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822522    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822525    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822527    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822530    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822533    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822536    2580 flags.go:64] FLAG: --v="2"
Jul 22 15:13:10.830434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822540    2580 flags.go:64] FLAG: --version="false"
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822546    2580 flags.go:64] FLAG: --vmodule=""
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822550    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822553    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822679    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822683    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822686    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822688    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822691    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822694    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822696    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822699    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822701    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822703    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822706    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822708    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822711    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822713    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822716    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822718    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:13:10.831092 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822721    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822724    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822729    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822732    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822734    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822737    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822739    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822742    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822744    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822747    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822752    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822755    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822758    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822762    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822765    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822767    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822770    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822773    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822775    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822778    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:13:10.831586 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822780    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822783    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822785    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822788    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822790    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822793    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822795    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822797    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822800    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822803    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822805    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822807    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822810    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822813    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822816    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822819    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822822    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822824    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822827    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822830    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:13:10.832164 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822832    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822835    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822837    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822839    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822842    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822846    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822848    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822851    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822853    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822855    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822858    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822860    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822863    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822865    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822868    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822870    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822873    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822875    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822878    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:13:10.832670 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822882    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822885    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822888    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822891    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822893    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822896    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822899    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822901    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822904    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822907    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822909    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822912    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822914    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822917    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822919    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822921    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822924    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822926    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822930    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:13:10.833139 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822933    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822936    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822939    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822941    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.822944    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.822950    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.830653    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.830673    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.830692    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.830698    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830772    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830777    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830780    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830783    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830786    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:13:10.833882 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830788    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830791    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830794    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830797    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830799    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830802    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830804    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830807    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830809    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830812    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830815    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830817    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830820    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830822    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830826    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830828    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830831    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830833    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830836    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830838    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:13:10.834507 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830841    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830843    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830846    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830848    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830857    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830860    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830862    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830865    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830867    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830870    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830872    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830875    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830877    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830879    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830882    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830884    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830887    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830891    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830895    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:13:10.835046 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830897    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830900    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830903    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830906    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830908    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830911    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830914    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830916    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830918    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830921    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830923    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830926    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830928    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830931    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830933    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830935    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830939    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830943    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830953    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:13:10.835679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830956    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830959    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830962    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830965    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830969    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830972    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830975    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830977    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830980    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830982    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830985    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830987    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830989    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830992    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830995    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.830997    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831000    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831002    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831005    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:13:10.836220 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831007    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831010    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831012    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831015    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831017    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831019    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831022    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831024    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831027    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831029    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831032    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831034    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831037    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831046    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831049    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831051    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:13:10.836836 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831054    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.831059    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831233    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831239    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831241    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831244    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831247    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831251    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831253    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831256    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831258    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831261    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831263    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831267    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:13:10.837237 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831270    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831272    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831275    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831277    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831279    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831282    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831285    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831287    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831290    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831292    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831295    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831297    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831299    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831302    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831311    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831313    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831321    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831324    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831327    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831329    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:13:10.837628 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831333    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831336    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831339    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831342    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831344    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831346    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831349    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831352    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831354    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831357    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831359    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831362    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831365    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831368    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831370    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831373    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831375    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831377    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831380    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831382    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:13:10.838180 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831384    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831387    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831389    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831391    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831394    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831397    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831399    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831402    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831404    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831412    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831415    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831417    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831420    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831422    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831425    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831427    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831430    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831432    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831434    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831437    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:13:10.838745 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831440    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831444    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831447    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831450    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831453    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831455    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831457    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831460    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831462    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831465    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831467    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831469    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831472    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831474    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831477    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831479    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831482    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831484    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831504    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:13:10.839251 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831509    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831512    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831515    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831524    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831527    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831530    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831532    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:10.831535    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.831539    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.831691    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.836524    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.837972    2580 server.go:1019] "Starting client certificate rotation"
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.838072    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 15:13:10.839762 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.838116    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 15:13:10.840767 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.840745    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 15:13:10.841998 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.841978    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 15:13:10.854110 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.854088    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 22 15:13:10.856600 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.856582    2580 log.go:25] "Validated CRI v1 image API"
Jul 22 15:13:10.858043 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.858027    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 15:13:10.862613 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.862592    2580 fs.go:135] Filesystem UUIDs: map[5e232106-22ca-4e06-ab4b-db31d0de3b00:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a0920c9c-f874-4603-8753-32b7d55113c6:/dev/nvme0n1p4]
Jul 22 15:13:10.862697 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.862612    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 15:13:10.866475 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.866453    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 15:13:10.869224 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.869111    2580 manager.go:216] Machine: {Timestamp:2026-07-22 15:13:10.867780563 +0000 UTC m=+0.309864001 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109575 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec260e90ef03d3f5aa3eb03b8a99e9b1 SystemUUID:ec260e90-ef03-d3f5-aa3e-b03b8a99e9b1 BootID:2e943d8f-fb45-4628-b809-1522a9542977 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:83:14:4c:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:83:14:4c:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:c4:00:2d:b6:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 15:13:10.869224 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.869217    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 15:13:10.869342 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.869312    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 15:13:10.872139 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.872102    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 15:13:10.872325 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.872143    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-231.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 15:13:10.872945 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.872932    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 15:13:10.872998 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.872951    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 15:13:10.872998 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.872962    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 15:13:10.873647 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.873634    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 15:13:10.875130 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.875118    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 15:13:10.879391 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879372    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 15:13:10.879564 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879552    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 15:13:10.879637 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879598    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 15:13:10.879637 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879613    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 15:13:10.879637 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879635    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 15:13:10.879754 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879650    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 22 15:13:10.879754 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.879664    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 15:13:10.880194 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.880156    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 15:13:10.880284 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.880196    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 15:13:10.884086 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.884060    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 15:13:10.886319 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.886304    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 15:13:10.886393 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.886343    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 15:13:10.887630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887617    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 15:13:10.887630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887633    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 15:13:10.887630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887639    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887645    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887654    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887659    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887667    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887672    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887679    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887685    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887694    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 15:13:10.887759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.887700    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 15:13:10.888371 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.888360    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 15:13:10.888371 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.888370    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 15:13:10.889098 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.889079    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4wc9l"
Jul 22 15:13:10.890406 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.890383    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-231.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 15:13:10.890499 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.890431    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 15:13:10.890499 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.890467    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 15:13:10.892021 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.892009    2580 server.go:1268] "Started kubelet"
Jul 22 15:13:10.892132 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.892088    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 15:13:10.892196 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.892156    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 15:13:10.892285 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.892247    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 15:13:10.892876 ip-10-0-143-231 systemd[1]: Started Kubernetes Kubelet.
Jul 22 15:13:10.894041 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.894025    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 15:13:10.894121 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.894025    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 15:13:10.896022 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.896004    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4wc9l"
Jul 22 15:13:10.897333 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.896474    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-231.ec2.internal.18c4a56195f73cdd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-231.ec2.internal,UID:ip-10-0-143-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-231.ec2.internal,},FirstTimestamp:2026-07-22 15:13:10.891977949 +0000 UTC m=+0.334061391,LastTimestamp:2026-07-22 15:13:10.891977949 +0000 UTC m=+0.334061391,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-231.ec2.internal,}"
Jul 22 15:13:10.897820 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.897802    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 15:13:10.902832 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.902252    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 15:13:10.902832 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.902527    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:10.902832 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.902696    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903000    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903016    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903033    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903124    2580 factory.go:55] Registering systemd factory
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903139    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903155    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 15:13:10.903238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903162    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 15:13:10.903579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903362    2580 factory.go:153] Registering CRI-O factory
Jul 22 15:13:10.903579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903376    2580 factory.go:223] Registration of the crio container factory successfully
Jul 22 15:13:10.903579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903426    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 15:13:10.903579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903447    2580 factory.go:103] Registering Raw factory
Jul 22 15:13:10.903579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.903463    2580 manager.go:1196] Started watching for new ooms in manager
Jul 22 15:13:10.906383 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.904240    2580 manager.go:349] Starting recovery of all containers
Jul 22 15:13:10.906383 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.905229    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 15:13:10.906383 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.905334    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 15:13:10.914969 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.914841    2580 manager.go:354] Recovery completed
Jul 22 15:13:10.919040 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.919024    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:10.923010 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.922991    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:10.923089 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.923024    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:10.923089 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.923034    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:10.923569 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.923549    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 15:13:10.923569 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.923565    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 15:13:10.923569 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.923571    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:10.923777 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.923585    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 15:13:10.924278 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.924260    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:10.924332 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.924292    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:10.924332 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.924302    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:10.925829 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.925815    2580 policy_none.go:49] "None policy: Start"
Jul 22 15:13:10.925898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.925835    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 15:13:10.925898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.925850    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 15:13:10.927834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.927819    2580 policy_none.go:47] "Start"
Jul 22 15:13:10.962172 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962146    2580 manager.go:337] "Starting Device Plugin manager"
Jul 22 15:13:10.962328 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.962201    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 15:13:10.962328 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962214    2580 server.go:87] "Starting device plugin registration server"
Jul 22 15:13:10.962544 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962530    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 15:13:10.962608 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962546    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 15:13:10.962651 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962614    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 15:13:10.962691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962678    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 15:13:10.962691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:10.962684    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 15:13:10.963327 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.963281    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 15:13:10.963410 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:10.963336    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.013308 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.013271    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 15:13:11.018981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.014660    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 15:13:11.018981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.014700    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 15:13:11.018981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.014723    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 15:13:11.018981 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.014765    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 15:13:11.018981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.017173    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:13:11.063190 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.063151    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:11.064184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.064158    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:11.064270 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.064195    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:11.064270 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.064213    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:11.064270 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.064238    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.072935 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.072876    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.072935 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.072905    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-231.ec2.internal\": node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.086561 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.086531    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.115010 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.114975    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"]
Jul 22 15:13:11.115164 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.115064    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:11.116118 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.116077    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:11.116118 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.116114    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:11.116244 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.116125    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:11.118362 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.118348    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:11.118466 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.118450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.118542 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.118480    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:11.119160 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.119139    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:11.119251 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.119170    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:11.119251 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.119176    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:11.119251 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.119188    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:11.119251 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.119193    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:11.119251 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.119203    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:11.121429 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.121411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.121536 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.121440    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:13:11.122198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.122182    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:13:11.122295 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.122211    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:13:11.122295 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.122224    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:13:11.141356 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.141330    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-231.ec2.internal\" not found" node="ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.146135 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.146116    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-231.ec2.internal\" not found" node="ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.186684 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.186650    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.205372 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.205329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3132d0a8bfd17a2c358c01f67e7fcefb-config\") pod \"kube-apiserver-proxy-ip-10-0-143-231.ec2.internal\" (UID: \"3132d0a8bfd17a2c358c01f67e7fcefb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.205372 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.205373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ef0226cd44ee0937ed191c077edfe2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal\" (UID: \"3ef0226cd44ee0937ed191c077edfe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.205536 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.205393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ef0226cd44ee0937ed191c077edfe2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal\" (UID: \"3ef0226cd44ee0937ed191c077edfe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.287438 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.287405    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.305845 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.305815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3132d0a8bfd17a2c358c01f67e7fcefb-config\") pod \"kube-apiserver-proxy-ip-10-0-143-231.ec2.internal\" (UID: \"3132d0a8bfd17a2c358c01f67e7fcefb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.305911 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.305848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3132d0a8bfd17a2c358c01f67e7fcefb-config\") pod \"kube-apiserver-proxy-ip-10-0-143-231.ec2.internal\" (UID: \"3132d0a8bfd17a2c358c01f67e7fcefb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.305911 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.305901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ef0226cd44ee0937ed191c077edfe2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal\" (UID: \"3ef0226cd44ee0937ed191c077edfe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.305976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.305918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ef0226cd44ee0937ed191c077edfe2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal\" (UID: \"3ef0226cd44ee0937ed191c077edfe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.305976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.305944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ef0226cd44ee0937ed191c077edfe2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal\" (UID: \"3ef0226cd44ee0937ed191c077edfe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.306033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.305979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ef0226cd44ee0937ed191c077edfe2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal\" (UID: \"3ef0226cd44ee0937ed191c077edfe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.388301 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.388216    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.445741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.445710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.449157 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.449136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:11.488923 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.488890    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.589456 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.589415    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.691117 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.691010    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.795360 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.795313    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.839151 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.839103    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 15:13:11.839711 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.839324    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 15:13:11.896087 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.896054    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:11.903014 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.902978    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 15:08:11 +0000 UTC" deadline="2028-04-06 21:32:34.86391867 +0000 UTC"
Jul 22 15:13:11.903088 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:11.903017    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14982h19m22.960907145s"
Jul 22 15:13:11.996724 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:11.996615    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:12.097809 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:12.097769    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:12.198822 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:12.198781    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-231.ec2.internal\" not found"
Jul 22 15:13:12.199721 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.199531    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:13:12.202951 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.202928    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:12.216975 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.216950    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 15:13:12.219315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.219296    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal"
Jul 22 15:13:12.229539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.229515    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 15:13:12.288235 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:12.288198    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef0226cd44ee0937ed191c077edfe2d.slice/crio-95c2e26d4c1559a7aadc94d016516b18fe5f71cae6202be7e8a56ea248fa07e8 WatchSource:0}: Error finding container 95c2e26d4c1559a7aadc94d016516b18fe5f71cae6202be7e8a56ea248fa07e8: Status 404 returned error can't find the container with id 95c2e26d4c1559a7aadc94d016516b18fe5f71cae6202be7e8a56ea248fa07e8
Jul 22 15:13:12.293767 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:12.293727    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3132d0a8bfd17a2c358c01f67e7fcefb.slice/crio-26b833c0ba20473422b7cee1f3dd1143d5683e58c489672296c7738c0af48ac1 WatchSource:0}: Error finding container 26b833c0ba20473422b7cee1f3dd1143d5683e58c489672296c7738c0af48ac1: Status 404 returned error can't find the container with id 26b833c0ba20473422b7cee1f3dd1143d5683e58c489672296c7738c0af48ac1
Jul 22 15:13:12.293853 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.293776    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:13:12.293980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.293966    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:13:12.435550 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.435492    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:13:12.884016 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.883822    2580 apiserver.go:52] "Watching apiserver"
Jul 22 15:13:12.892259 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.892227    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 15:13:12.892777 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.892732    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4lwsx","kube-system/konnectivity-agent-g4wqx","kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal","openshift-dns/node-resolver-hrk9j","openshift-image-registry/node-ca-7zmpm","openshift-multus/multus-g6msh","openshift-multus/network-metrics-daemon-58d6n","openshift-network-diagnostics/network-check-target-28qwf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p","openshift-cluster-node-tuning-operator/tuned-7qvc6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal","openshift-multus/multus-additional-cni-plugins-c9vpv","openshift-network-operator/iptables-alerter-9xcfs","openshift-ovn-kubernetes/ovnkube-node-h82fj"]
Jul 22 15:13:12.895439 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.895410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:12.895604 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:12.895517    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:12.898294 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.898272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:12.902236 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.901261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.902236 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.901791    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.902413 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.902393    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kgdm9\""
Jul 22 15:13:12.902984 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.902654    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.902984 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.902855    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 15:13:12.903225 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.903201    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 15:13:12.903905 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.903674    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.904208 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.904188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:12.904449 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.904431    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 15:13:12.904912 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.904733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s6tnt\""
Jul 22 15:13:12.904912 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.904816    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 15:13:12.905497 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.905305    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.906498 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.906473    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 15:13:12.909431 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.906749    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.909431 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.906828    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 15:13:12.909431 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.907093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:12.911267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.910686    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z9fpd\""
Jul 22 15:13:12.911267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.910698    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.911267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.911012    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.911267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.911091    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 15:13:12.911517 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.911281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2db9b\""
Jul 22 15:13:12.911517 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.911383    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.916099 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.912884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g6msh"
Jul 22 15:13:12.916099 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.915417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.916099 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.915875    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 15:13:12.916282 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916131    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 15:13:12.916328 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kkwkg\""
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxq96\" (UniqueName: \"kubernetes.io/projected/c259c676-391f-4797-a0f6-95f7e56b83b6-kube-api-access-jxq96\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916605    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9409f52c-ec04-4289-9cec-5afdb0a72d73-host-slash\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovnkube-config\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-env-overrides\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovnkube-script-lib\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-tmp-dir\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916824    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln9gp\" (UniqueName: \"kubernetes.io/projected/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-kube-api-access-ln9gp\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-598lv\" (UniqueName: \"kubernetes.io/projected/03c418d8-505d-4ea6-92da-18ae9bba17e5-kube-api-access-598lv\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.916910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78294\" (UniqueName: \"kubernetes.io/projected/9409f52c-ec04-4289-9cec-5afdb0a72d73-kube-api-access-78294\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-kubelet\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.916947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-slash\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-ovn\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-cni-netd\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9409f52c-ec04-4289-9cec-5afdb0a72d73-iptables-alerter-script\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-cni-bin\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovn-node-metrics-cert\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-hosts-file\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-systemd\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-etc-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-node-log\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-log-socket\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.917516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-systemd-units\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.918405 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-run-netns\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.918405 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.917378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-var-lib-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.918991    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.919538    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.919911    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.919972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:12.920043    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.920155    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.920440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.920266    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pc9v4\""
Jul 22 15:13:12.922096 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.922074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:12.926259 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.926238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:12.929067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.928611    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 15:13:12.929067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.928833    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 15:13:12.929067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.928844    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-27c8b\""
Jul 22 15:13:12.931730 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.929776    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:13:12.931730 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.930122    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bgm58\""
Jul 22 15:13:12.931730 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.930131    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 15:13:12.934199 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.933839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:12.934199 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:12.934060    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:12.937393 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.937369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:12.938491 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.938468    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rdngd"
Jul 22 15:13:12.940263 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.939936    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 15:13:12.940263 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.939969    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 15:13:12.940263 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.939979    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vzwgv\""
Jul 22 15:13:12.944433 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:12.944411    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rdngd"
Jul 22 15:13:13.004367 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.004335    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 15:13:13.017669 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-dbus\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.017669 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42tfv\" (UniqueName: \"kubernetes.io/projected/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-kube-api-access-42tfv\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.017899 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m82rj\" (UniqueName: \"kubernetes.io/projected/1a043e63-87db-4da7-8fbb-0f13536fcc38-kube-api-access-m82rj\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.017899 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-registration-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.017899 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-systemd\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.017899 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-systemd\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-node-log\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-hostroot\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-node-log\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.017997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-systemd-units\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-run-netns\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-systemd-units\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-var-lib-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-host\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-run-netns\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.018162 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-var-lib-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-modprobe-d\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-var-lib-kubelet\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-system-cni-dir\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-socket-dir-parent\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-cni-bin\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-daemon-config\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hm7t\" (UniqueName: \"kubernetes.io/projected/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-kube-api-access-9hm7t\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovnkube-config\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkdh5\" (UniqueName: \"kubernetes.io/projected/61a78276-6988-4d12-a212-7d51a6640386-kube-api-access-rkdh5\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-etc-selinux\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-k8s-cni-cncf-io\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovnkube-script-lib\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-tmp-dir\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cnibin\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.018768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-cnibin\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-socket-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-598lv\" (UniqueName: \"kubernetes.io/projected/03c418d8-505d-4ea6-92da-18ae9bba17e5-kube-api-access-598lv\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-kubelet\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-hosts-file\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysconfig\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.018984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-system-cni-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-tmp-dir\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-cni-bin\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61a78276-6988-4d12-a212-7d51a6640386-tmp\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.019127    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:13.019589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-cni-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovnkube-config\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-etc-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-kubelet\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-etc-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.019228    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:13.519188548 +0000 UTC m=+2.961271982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-cni-bin\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-openvswitch\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-hosts-file\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-log-socket\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysctl-d\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61a78276-6988-4d12-a212-7d51a6640386-etc-tuned\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-log-socket\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-sys-fs\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8wrq\" (UniqueName: \"kubernetes.io/projected/f444fe3e-64c6-4c53-a3dd-74548e58e715-kube-api-access-d8wrq\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.020453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1aa3111a-a405-44fc-96b5-d951e0d8f3ce-konnectivity-ca\") pod \"konnectivity-agent-g4wqx\" (UID: \"1aa3111a-a405-44fc-96b5-d951e0d8f3ce\") " pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-kubelet-config\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-kubernetes\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a043e63-87db-4da7-8fbb-0f13536fcc38-serviceca\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-multus-certs\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019833    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovnkube-script-lib\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysctl-conf\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-sys\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-netns\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-kubelet\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-etc-kubernetes\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.019990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-device-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxq96\" (UniqueName: \"kubernetes.io/projected/c259c676-391f-4797-a0f6-95f7e56b83b6-kube-api-access-jxq96\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9409f52c-ec04-4289-9cec-5afdb0a72d73-host-slash\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-lib-modules\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a043e63-87db-4da7-8fbb-0f13536fcc38-host\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.021120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9409f52c-ec04-4289-9cec-5afdb0a72d73-host-slash\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-cni-binary-copy\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-run\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-cni-multus\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-conf-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-env-overrides\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln9gp\" (UniqueName: \"kubernetes.io/projected/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-kube-api-access-ln9gp\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1aa3111a-a405-44fc-96b5-d951e0d8f3ce-agent-certs\") pod \"konnectivity-agent-g4wqx\" (UID: \"1aa3111a-a405-44fc-96b5-d951e0d8f3ce\") " pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78294\" (UniqueName: \"kubernetes.io/projected/9409f52c-ec04-4289-9cec-5afdb0a72d73-kube-api-access-78294\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-slash\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-ovn\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-cni-netd\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovn-node-metrics-cert\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-systemd\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-os-release\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.021817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-os-release\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9409f52c-ec04-4289-9cec-5afdb0a72d73-iptables-alerter-script\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-cni-netd\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-host-slash\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03c418d8-505d-4ea6-92da-18ae9bba17e5-run-ovn\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.020802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03c418d8-505d-4ea6-92da-18ae9bba17e5-env-overrides\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.021160    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.021183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9409f52c-ec04-4289-9cec-5afdb0a72d73-iptables-alerter-script\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.022436 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.021501    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal" event={"ID":"3132d0a8bfd17a2c358c01f67e7fcefb","Type":"ContainerStarted","Data":"26b833c0ba20473422b7cee1f3dd1143d5683e58c489672296c7738c0af48ac1"}
Jul 22 15:13:13.023138 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.023110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal" event={"ID":"3ef0226cd44ee0937ed191c077edfe2d","Type":"ContainerStarted","Data":"95c2e26d4c1559a7aadc94d016516b18fe5f71cae6202be7e8a56ea248fa07e8"}
Jul 22 15:13:13.025374 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.025349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03c418d8-505d-4ea6-92da-18ae9bba17e5-ovn-node-metrics-cert\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.034118 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.034084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-598lv\" (UniqueName: \"kubernetes.io/projected/03c418d8-505d-4ea6-92da-18ae9bba17e5-kube-api-access-598lv\") pod \"ovnkube-node-h82fj\" (UID: \"03c418d8-505d-4ea6-92da-18ae9bba17e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.035073 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.035047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln9gp\" (UniqueName: \"kubernetes.io/projected/c471fe40-e8a7-4c30-bf41-f3bcbe56d275-kube-api-access-ln9gp\") pod \"node-resolver-hrk9j\" (UID: \"c471fe40-e8a7-4c30-bf41-f3bcbe56d275\") " pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.036424 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.036371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxq96\" (UniqueName: \"kubernetes.io/projected/c259c676-391f-4797-a0f6-95f7e56b83b6-kube-api-access-jxq96\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:13.037152 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.037108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78294\" (UniqueName: \"kubernetes.io/projected/9409f52c-ec04-4289-9cec-5afdb0a72d73-kube-api-access-78294\") pod \"iptables-alerter-9xcfs\" (UID: \"9409f52c-ec04-4289-9cec-5afdb0a72d73\") " pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-run\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-cni-multus\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-conf-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1aa3111a-a405-44fc-96b5-d951e0d8f3ce-agent-certs\") pod \"konnectivity-agent-g4wqx\" (UID: \"1aa3111a-a405-44fc-96b5-d951e0d8f3ce\") " pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-systemd\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-os-release\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-os-release\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-dbus\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42tfv\" (UniqueName: \"kubernetes.io/projected/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-kube-api-access-42tfv\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m82rj\" (UniqueName: \"kubernetes.io/projected/1a043e63-87db-4da7-8fbb-0f13536fcc38-kube-api-access-m82rj\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-registration-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-hostroot\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-host\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.124403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-modprobe-d\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-var-lib-kubelet\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-system-cni-dir\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-socket-dir-parent\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-cni-bin\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.121995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-daemon-config\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hm7t\" (UniqueName: \"kubernetes.io/projected/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-kube-api-access-9hm7t\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkdh5\" (UniqueName: \"kubernetes.io/projected/61a78276-6988-4d12-a212-7d51a6640386-kube-api-access-rkdh5\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-etc-selinux\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-k8s-cni-cncf-io\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cnibin\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-cnibin\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-socket-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysconfig\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.125465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-system-cni-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61a78276-6988-4d12-a212-7d51a6640386-tmp\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-cni-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysctl-d\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61a78276-6988-4d12-a212-7d51a6640386-etc-tuned\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-sys-fs\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wrq\" (UniqueName: \"kubernetes.io/projected/f444fe3e-64c6-4c53-a3dd-74548e58e715-kube-api-access-d8wrq\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.122911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-system-cni-dir\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-dbus\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-cni-bin\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-run\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-registration-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-daemon-config\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123478    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-cni-multus\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-conf-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.126280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-modprobe-d\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.123972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1aa3111a-a405-44fc-96b5-d951e0d8f3ce-konnectivity-ca\") pod \"konnectivity-agent-g4wqx\" (UID: \"1aa3111a-a405-44fc-96b5-d951e0d8f3ce\") " pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-kubelet-config\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-kubernetes\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a043e63-87db-4da7-8fbb-0f13536fcc38-serviceca\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-etc-selinux\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-multus-certs\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysctl-conf\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-k8s-cni-cncf-io\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-sys\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-netns\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-cnibin\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-netns\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-kubelet\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-etc-kubernetes\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-cnibin\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.128534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-device-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-lib-modules\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-socket-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a043e63-87db-4da7-8fbb-0f13536fcc38-host\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.124501    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.124579    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:13.624546938 +0000 UTC m=+3.066630366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a043e63-87db-4da7-8fbb-0f13536fcc38-host\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-hostroot\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-host\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-var-lib-kubelet\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-etc-kubernetes\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.124965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysconfig\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-system-cni-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-device-dir\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-cni-dir\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.129446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f444fe3e-64c6-4c53-a3dd-74548e58e715-sys-fs\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysctl-d\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-var-lib-kubelet\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1aa3111a-a405-44fc-96b5-d951e0d8f3ce-konnectivity-ca\") pod \"konnectivity-agent-g4wqx\" (UID: \"1aa3111a-a405-44fc-96b5-d951e0d8f3ce\") " pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-cni-binary-copy\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-kubelet-config\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-kubernetes\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-lib-modules\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a043e63-87db-4da7-8fbb-0f13536fcc38-serviceca\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.125938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-host-run-multus-certs\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-sysctl-conf\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-sys\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-cni-binary-copy\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-os-release\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61a78276-6988-4d12-a212-7d51a6640386-etc-systemd\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-os-release\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.126677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-multus-socket-dir-parent\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.128856    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.128889    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:13.130382 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.128905    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:13.131415 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.128995    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:13.628976247 +0000 UTC m=+3.071059693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:13.131415 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.129920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61a78276-6988-4d12-a212-7d51a6640386-tmp\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.133319 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.133289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1aa3111a-a405-44fc-96b5-d951e0d8f3ce-agent-certs\") pod \"konnectivity-agent-g4wqx\" (UID: \"1aa3111a-a405-44fc-96b5-d951e0d8f3ce\") " pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.136151 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.136087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42tfv\" (UniqueName: \"kubernetes.io/projected/0f7f3a35-79f8-491e-82d2-f4cd36f14c4a-kube-api-access-42tfv\") pod \"multus-additional-cni-plugins-c9vpv\" (UID: \"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a\") " pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.137929 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.137902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hm7t\" (UniqueName: \"kubernetes.io/projected/82d4e7bf-c677-4546-b439-9dd43dfb1e5d-kube-api-access-9hm7t\") pod \"multus-g6msh\" (UID: \"82d4e7bf-c677-4546-b439-9dd43dfb1e5d\") " pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.138033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.137914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61a78276-6988-4d12-a212-7d51a6640386-etc-tuned\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.138622 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.138596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m82rj\" (UniqueName: \"kubernetes.io/projected/1a043e63-87db-4da7-8fbb-0f13536fcc38-kube-api-access-m82rj\") pod \"node-ca-7zmpm\" (UID: \"1a043e63-87db-4da7-8fbb-0f13536fcc38\") " pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.140876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.140848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkdh5\" (UniqueName: \"kubernetes.io/projected/61a78276-6988-4d12-a212-7d51a6640386-kube-api-access-rkdh5\") pod \"tuned-7qvc6\" (UID: \"61a78276-6988-4d12-a212-7d51a6640386\") " pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.144031 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.144011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8wrq\" (UniqueName: \"kubernetes.io/projected/f444fe3e-64c6-4c53-a3dd-74548e58e715-kube-api-access-d8wrq\") pod \"aws-ebs-csi-driver-node-zdd5p\" (UID: \"f444fe3e-64c6-4c53-a3dd-74548e58e715\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.217043 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.216988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9xcfs"
Jul 22 15:13:13.228539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.228508    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:13.232134 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.232100    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9409f52c_ec04_4289_9cec_5afdb0a72d73.slice/crio-b01418d960272fd9c150035044e990ff13fffdaa7c1177317157dbbe10a953db WatchSource:0}: Error finding container b01418d960272fd9c150035044e990ff13fffdaa7c1177317157dbbe10a953db: Status 404 returned error can't find the container with id b01418d960272fd9c150035044e990ff13fffdaa7c1177317157dbbe10a953db
Jul 22 15:13:13.238855 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.238826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hrk9j"
Jul 22 15:13:13.243588 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.242116    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c418d8_505d_4ea6_92da_18ae9bba17e5.slice/crio-1bfc222e6acc87c5f144515cd4e5183f769361f72c4fa1a6f53b989152447d7a WatchSource:0}: Error finding container 1bfc222e6acc87c5f144515cd4e5183f769361f72c4fa1a6f53b989152447d7a: Status 404 returned error can't find the container with id 1bfc222e6acc87c5f144515cd4e5183f769361f72c4fa1a6f53b989152447d7a
Jul 22 15:13:13.246736 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.246659    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:13:13.248137 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.248111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7zmpm"
Jul 22 15:13:13.256548 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.256417    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc471fe40_e8a7_4c30_bf41_f3bcbe56d275.slice/crio-fb7ab2f85ffaf7c4a9e85249a259ab8a18c64e2a638722d1f781b3047db3940d WatchSource:0}: Error finding container fb7ab2f85ffaf7c4a9e85249a259ab8a18c64e2a638722d1f781b3047db3940d: Status 404 returned error can't find the container with id fb7ab2f85ffaf7c4a9e85249a259ab8a18c64e2a638722d1f781b3047db3940d
Jul 22 15:13:13.258790 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.258760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g6msh"
Jul 22 15:13:13.268838 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.268760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p"
Jul 22 15:13:13.278041 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.277387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:13.278374 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.278346    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d4e7bf_c677_4546_b439_9dd43dfb1e5d.slice/crio-acf1ea7975cfbb77982fa5b7239ca74cfc4cf21b15842b67a05cf78126ea1a18 WatchSource:0}: Error finding container acf1ea7975cfbb77982fa5b7239ca74cfc4cf21b15842b67a05cf78126ea1a18: Status 404 returned error can't find the container with id acf1ea7975cfbb77982fa5b7239ca74cfc4cf21b15842b67a05cf78126ea1a18
Jul 22 15:13:13.286736 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.286713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7qvc6"
Jul 22 15:13:13.291236 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.291083    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa3111a_a405_44fc_96b5_d951e0d8f3ce.slice/crio-afcb41059247a3c0a28529ac2afbfcd90b05cbc7d1c584d0095c27cf2484d9f0 WatchSource:0}: Error finding container afcb41059247a3c0a28529ac2afbfcd90b05cbc7d1c584d0095c27cf2484d9f0: Status 404 returned error can't find the container with id afcb41059247a3c0a28529ac2afbfcd90b05cbc7d1c584d0095c27cf2484d9f0
Jul 22 15:13:13.295632 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.295599    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c9vpv"
Jul 22 15:13:13.299980 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.299822    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a78276_6988_4d12_a212_7d51a6640386.slice/crio-48da76a0c54adc7eb5a2394298756e61bd141b8f7f4cfdcceabfa5f32be72bb2 WatchSource:0}: Error finding container 48da76a0c54adc7eb5a2394298756e61bd141b8f7f4cfdcceabfa5f32be72bb2: Status 404 returned error can't find the container with id 48da76a0c54adc7eb5a2394298756e61bd141b8f7f4cfdcceabfa5f32be72bb2
Jul 22 15:13:13.309679 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:13:13.309646    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7f3a35_79f8_491e_82d2_f4cd36f14c4a.slice/crio-d7ed729fc97d477f072d89f0080108c5041071b5dc337ec1abdbe46c9fb21490 WatchSource:0}: Error finding container d7ed729fc97d477f072d89f0080108c5041071b5dc337ec1abdbe46c9fb21490: Status 404 returned error can't find the container with id d7ed729fc97d477f072d89f0080108c5041071b5dc337ec1abdbe46c9fb21490
Jul 22 15:13:13.527788 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.527683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:13.527970 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.527874    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:13.528116 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.527973    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:14.527942177 +0000 UTC m=+3.970025620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:13.628565 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.628504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:13.628757 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.628719    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:13.628819 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.628807    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:14.628785079 +0000 UTC m=+4.070868521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:13.729787 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.729750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:13.729977 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.729955    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:13.730037 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.729984    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:13.730037 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.729995    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:13.730127 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:13.730057    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:14.730037647 +0000 UTC m=+4.172121074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:13.947961 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.947861    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 15:08:13 +0000 UTC" deadline="2028-04-12 15:54:32.47783671 +0000 UTC"
Jul 22 15:13:13.947961 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:13.947905    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15120h41m18.529936861s"
Jul 22 15:13:14.028342 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.028191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4wqx" event={"ID":"1aa3111a-a405-44fc-96b5-d951e0d8f3ce","Type":"ContainerStarted","Data":"afcb41059247a3c0a28529ac2afbfcd90b05cbc7d1c584d0095c27cf2484d9f0"}
Jul 22 15:13:14.031910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.031825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g6msh" event={"ID":"82d4e7bf-c677-4546-b439-9dd43dfb1e5d","Type":"ContainerStarted","Data":"acf1ea7975cfbb77982fa5b7239ca74cfc4cf21b15842b67a05cf78126ea1a18"}
Jul 22 15:13:14.034135 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.034037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7zmpm" event={"ID":"1a043e63-87db-4da7-8fbb-0f13536fcc38","Type":"ContainerStarted","Data":"3ed69e86bbe5ca656988bb4f616d922c9492de1eaae0bef39d58752bd9c963b8"}
Jul 22 15:13:14.036121 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.036040    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hrk9j" event={"ID":"c471fe40-e8a7-4c30-bf41-f3bcbe56d275","Type":"ContainerStarted","Data":"fb7ab2f85ffaf7c4a9e85249a259ab8a18c64e2a638722d1f781b3047db3940d"}
Jul 22 15:13:14.037945 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.037859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9xcfs" event={"ID":"9409f52c-ec04-4289-9cec-5afdb0a72d73","Type":"ContainerStarted","Data":"b01418d960272fd9c150035044e990ff13fffdaa7c1177317157dbbe10a953db"}
Jul 22 15:13:14.040195 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.040157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerStarted","Data":"d7ed729fc97d477f072d89f0080108c5041071b5dc337ec1abdbe46c9fb21490"}
Jul 22 15:13:14.043118 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.043084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7qvc6" event={"ID":"61a78276-6988-4d12-a212-7d51a6640386","Type":"ContainerStarted","Data":"48da76a0c54adc7eb5a2394298756e61bd141b8f7f4cfdcceabfa5f32be72bb2"}
Jul 22 15:13:14.046938 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.046911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p" event={"ID":"f444fe3e-64c6-4c53-a3dd-74548e58e715","Type":"ContainerStarted","Data":"5e2f774c8e1b91ded28713e59d25cfbb29732f6cb8ca01e956a25602d7239f37"}
Jul 22 15:13:14.050676 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.050643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"1bfc222e6acc87c5f144515cd4e5183f769361f72c4fa1a6f53b989152447d7a"}
Jul 22 15:13:14.534715 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.534673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:14.534922 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.534879    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:14.534988 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.534970    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:16.534947157 +0000 UTC m=+5.977030590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:14.635958 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.635278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:14.635958 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.635498    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:14.635958 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.635601    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:16.635578662 +0000 UTC m=+6.077662111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:14.736795 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.736749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:14.737048 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.737026    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:14.737130 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.737058    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:14.737130 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.737073    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:14.737251 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:14.737137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:16.737117588 +0000 UTC m=+6.179201034 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:14.948194 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.948086    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 15:08:13 +0000 UTC" deadline="2028-04-02 07:44:01.92778741 +0000 UTC"
Jul 22 15:13:14.948194 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:14.948137    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14872h30m46.979655584s"
Jul 22 15:13:15.017992 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:15.017958    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:15.018184 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:15.018110    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:15.018579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:15.018548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:15.018719 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:15.018695    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:15.018814 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:15.018776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:15.018871 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:15.018857    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:16.065670 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.065379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal" event={"ID":"3132d0a8bfd17a2c358c01f67e7fcefb","Type":"ContainerStarted","Data":"65b749b6235a3d413789267883153b8fcc07edf39ece6a59bd4e144e4b9aa796"}
Jul 22 15:13:16.073033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.072946    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ef0226cd44ee0937ed191c077edfe2d" containerID="6fac3c04fde490a5a3fcb6cf41e9b5af5ad58d88d2b2fa36600d994d7d4409e3" exitCode=0
Jul 22 15:13:16.073033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.073006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal" event={"ID":"3ef0226cd44ee0937ed191c077edfe2d","Type":"ContainerDied","Data":"6fac3c04fde490a5a3fcb6cf41e9b5af5ad58d88d2b2fa36600d994d7d4409e3"}
Jul 22 15:13:16.081141 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.081080    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-231.ec2.internal" podStartSLOduration=4.081059946 podStartE2EDuration="4.081059946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:16.080945001 +0000 UTC m=+5.523028448" watchObservedRunningTime="2026-07-22 15:13:16.081059946 +0000 UTC m=+5.523143395"
Jul 22 15:13:16.555297 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.555206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:16.555441 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.555402    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:16.555534 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.555515    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:20.555477002 +0000 UTC m=+9.997560463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:16.656549 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.656509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:16.656746 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.656694    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:16.656807 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.656758    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:20.656739695 +0000 UTC m=+10.098823125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:16.757069 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:16.757033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:16.757258 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.757213    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:16.757258 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.757234    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:16.757258 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.757248    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:16.757421 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:16.757312    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:20.757292982 +0000 UTC m=+10.199376422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:17.017802 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:17.017718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:17.017958 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:17.017844    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:17.018256 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:17.018235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:17.018392 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:17.018340    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:17.018473 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:17.018405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:17.018561 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:17.018482    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:19.015882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:19.015844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:19.016418 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:19.015986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:19.016418 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:19.016274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:19.016418 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:19.016354    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:19.016618 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:19.016511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:19.016673 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:19.016622    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:20.589447 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:20.589399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:20.589943 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.589566    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:20.589943 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.589637    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:28.589615938 +0000 UTC m=+18.031699367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:20.689800 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:20.689751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:20.690002 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.689931    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:20.690066 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.690007    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:28.68998609 +0000 UTC m=+18.132069529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:20.790226 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:20.790179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:20.790429 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.790413    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:20.790514 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.790435    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:20.790514 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.790449    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:20.790514 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:20.790516    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:28.790499835 +0000 UTC m=+18.232583274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:21.016617 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:21.016523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:21.016790 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:21.016632    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:21.017026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:21.017007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:21.017135 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:21.017116    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:21.017203 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:21.017186    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:21.017293 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:21.017276    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:23.015712 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:23.015672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:23.016177 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:23.015727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:23.016177 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:23.015688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:23.016177 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:23.015860    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:23.016177 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:23.015932    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:23.016177 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:23.016028    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:25.015881 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:25.015842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:25.016280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:25.015844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:25.016280 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:25.015954    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:25.016280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:25.015842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:25.016280 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:25.016042    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:25.016280 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:25.016095    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:27.015115 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:27.015071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:27.015655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:27.015076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:27.015655 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:27.015228    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:27.015655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:27.015076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:27.015655 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:27.015320    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:27.015655 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:27.015357    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:28.652836 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:28.652789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:28.653386 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.652996    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:28.653386 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.653065    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:44.653045636 +0000 UTC m=+34.095129061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:28.753716 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:28.753682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:28.753892 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.753822    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:28.753892 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.753892    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:44.753876146 +0000 UTC m=+34.195959572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:28.854092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:28.854049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:28.854289 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.854244    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:28.854289 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.854270    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:28.854289 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.854282    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:28.854434 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:28.854350    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:13:44.854330478 +0000 UTC m=+34.296413918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:29.015466 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:29.015433    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:29.015466 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:29.015471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:29.015722 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:29.015442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:29.015722 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:29.015576    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:29.015722 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:29.015697    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:29.015971 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:29.015795    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:31.017267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:31.016771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:31.017267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:31.016886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:31.017267 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:31.016886    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:31.017267 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:31.016948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:31.017267 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:31.017029    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:31.017267 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:31.017102    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:33.018480 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:33.018448    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:33.019031 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:33.018513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:33.019031 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:33.018514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:33.019031 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:33.018599    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:33.019031 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:33.018714    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:33.019031 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:33.018805    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:34.112303 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.111862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7zmpm" event={"ID":"1a043e63-87db-4da7-8fbb-0f13536fcc38","Type":"ContainerStarted","Data":"006ab10ac82ae232a5dadd92b66bfa0fc053ea061c1e43ed756ecba1a87074cc"}
Jul 22 15:13:34.113619 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.113589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hrk9j" event={"ID":"c471fe40-e8a7-4c30-bf41-f3bcbe56d275","Type":"ContainerStarted","Data":"2895c982cc1e050c00e21e10f32a57bb5829508c6fe1598b4c0102647fad1f5e"}
Jul 22 15:13:34.115347 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.115299    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7f3a35-79f8-491e-82d2-f4cd36f14c4a" containerID="17236188f8283358e3ff80e70bee08b646220a24b6186a9e20791d9a778fe70d" exitCode=0
Jul 22 15:13:34.115453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.115383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerDied","Data":"17236188f8283358e3ff80e70bee08b646220a24b6186a9e20791d9a778fe70d"}
Jul 22 15:13:34.117047 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.116856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7qvc6" event={"ID":"61a78276-6988-4d12-a212-7d51a6640386","Type":"ContainerStarted","Data":"bffb63d85274000b7b57bc80e8dcb1cca285f1b0539dee783cea0732087c1c41"}
Jul 22 15:13:34.118748 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.118639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p" event={"ID":"f444fe3e-64c6-4c53-a3dd-74548e58e715","Type":"ContainerStarted","Data":"788ba2aae3daecc1271a902560ab4c7b34e3355fc37479e81f051108120fb619"}
Jul 22 15:13:34.121105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.121073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"1112d64c75dc40a285fe227aa0e568635df229e1ef797b48b2bf0bbaef0f4116"}
Jul 22 15:13:34.121198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.121145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"45d7b93b0a2ea5e47561aa0c6e86fc4e3985db80083f163d2fb84aa26453854e"}
Jul 22 15:13:34.121198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.121188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"652c9a619a0d3c37871305a8ac9f45bcc673819aae1fe4b74e9af4aa2669cc8c"}
Jul 22 15:13:34.122868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.122831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal" event={"ID":"3ef0226cd44ee0937ed191c077edfe2d","Type":"ContainerStarted","Data":"620d01b3287d5741a61072f34255d9424eb7c1d2caac25639fec64c5ffe7c58e"}
Jul 22 15:13:34.124400 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.124377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4wqx" event={"ID":"1aa3111a-a405-44fc-96b5-d951e0d8f3ce","Type":"ContainerStarted","Data":"3ab4732c99415ec071c8254ce12692d7d55d357f6686956d47bfcabd3b1c8c2f"}
Jul 22 15:13:34.125746 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.125726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g6msh" event={"ID":"82d4e7bf-c677-4546-b439-9dd43dfb1e5d","Type":"ContainerStarted","Data":"b6c682bf2dfd724155701c1af960e3cdc3fd1a3785b2f4d80c03d5dae4139e4c"}
Jul 22 15:13:34.131435 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.131382    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7zmpm" podStartSLOduration=2.924959121 podStartE2EDuration="23.131364053s" totalImagesPullingTime="20.206404932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:34.130347524 +0000 UTC m=+23.572430965" watchObservedRunningTime="2026-07-22 15:13:34.131364053 +0000 UTC m=+23.573447500"
Jul 22 15:13:34.147937 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.147886    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g6msh" podStartSLOduration=2.886910596 podStartE2EDuration="23.147870092s" totalImagesPullingTime="20.260959496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:34.147623397 +0000 UTC m=+23.589706844" watchObservedRunningTime="2026-07-22 15:13:34.147870092 +0000 UTC m=+23.589953530"
Jul 22 15:13:34.163647 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.163600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g4wqx" podStartSLOduration=2.990779258 podStartE2EDuration="23.163583289s" totalImagesPullingTime="20.172804031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:34.163148718 +0000 UTC m=+23.605232165" watchObservedRunningTime="2026-07-22 15:13:34.163583289 +0000 UTC m=+23.605666780"
Jul 22 15:13:34.201698 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.201644    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hrk9j" podStartSLOduration=2.984175704 podStartE2EDuration="23.20162794s" totalImagesPullingTime="20.217452236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:34.201530325 +0000 UTC m=+23.643613772" watchObservedRunningTime="2026-07-22 15:13:34.20162794 +0000 UTC m=+23.643711387"
Jul 22 15:13:34.221262 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.221208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7qvc6" podStartSLOduration=3.040227106 podStartE2EDuration="23.221193779s" totalImagesPullingTime="20.180966673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:34.220700608 +0000 UTC m=+23.662784055" watchObservedRunningTime="2026-07-22 15:13:34.221193779 +0000 UTC m=+23.663277225"
Jul 22 15:13:34.237387 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:34.237345    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-231.ec2.internal" podStartSLOduration=22.237324854 podStartE2EDuration="22.237324854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:34.236849883 +0000 UTC m=+23.678933332" watchObservedRunningTime="2026-07-22 15:13:34.237324854 +0000 UTC m=+23.679408300"
Jul 22 15:13:35.015294 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.015255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:35.015545 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.015268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:35.015545 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:35.015389    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:35.015545 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.015400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:35.015545 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:35.015481    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:35.015720 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:35.015622    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:35.130623 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.130545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9xcfs" event={"ID":"9409f52c-ec04-4289-9cec-5afdb0a72d73","Type":"ContainerStarted","Data":"b108bde99cd96552a43d5cc90849b84d3af1a664cc703f3cba7565a9a7618a6c"}
Jul 22 15:13:35.134120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.134086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"92f7bff4f7f9b31ceada8a581112c42cf4f124d259e36b36a9fa04b79c81bba0"}
Jul 22 15:13:35.134274 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.134171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"3d25d55f5fbb9dafa93f1ddc5082cf4f95b59993d2851f98740c5dbf57613e1b"}
Jul 22 15:13:35.134274 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.134220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"42aa8105f69c914c5802de9f8a3dad9f6e0c69adbfc000b5fb1523ff7121a696"}
Jul 22 15:13:35.355951 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.355914    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 15:13:35.978439 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.978321    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T15:13:35.35596789Z","UUID":"9df16b6b-b65d-412c-9cf8-1be31151de37","Handler":null,"Name":"","Endpoint":""}
Jul 22 15:13:35.980273 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.980245    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 15:13:35.980473 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:35.980289    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 15:13:36.139217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:36.139175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p" event={"ID":"f444fe3e-64c6-4c53-a3dd-74548e58e715","Type":"ContainerStarted","Data":"e9213ef04631964c16d8efcd2773e1c5af875cab17d7ab5a8177065654f9b724"}
Jul 22 15:13:37.015853 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.015815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:37.016031 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.015816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:37.016031 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:37.015929    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:37.016031 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.015821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:37.016031 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:37.016010    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:37.016167 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:37.016099    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:37.145650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.145605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"715f3a7276ee97515c9150061ba955b0dff9fdfd00eacd813060aed5d40be8d4"}
Jul 22 15:13:37.848976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.848737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:37.849501 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.849460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:37.864689 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:37.864614    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9xcfs" podStartSLOduration=6.621793751 podStartE2EDuration="26.86459696s" totalImagesPullingTime="20.242803209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:35.144514685 +0000 UTC m=+24.586598128" watchObservedRunningTime="2026-07-22 15:13:37.86459696 +0000 UTC m=+27.306680470"
Jul 22 15:13:38.147800 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:38.147719    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:38.148311 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:38.148296    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g4wqx"
Jul 22 15:13:39.015105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.015054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:39.015105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.015094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:39.015213 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.015058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:39.015213 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:39.015172    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:39.015611 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:39.015268    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:39.015611 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:39.015374    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:39.152363 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.152087    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7f3a35-79f8-491e-82d2-f4cd36f14c4a" containerID="87f0184f3806f756812647e7cfdebb57804c6fcb893458eb41e721706e5f4916" exitCode=0
Jul 22 15:13:39.152363 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.152112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerDied","Data":"87f0184f3806f756812647e7cfdebb57804c6fcb893458eb41e721706e5f4916"}
Jul 22 15:13:39.158000 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.157971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p" event={"ID":"f444fe3e-64c6-4c53-a3dd-74548e58e715","Type":"ContainerStarted","Data":"6a3d9474ef7c7d20112c1595ba3ad1d7e05875ed1a2dcc3effe8caccd0426fcf"}
Jul 22 15:13:39.161768 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.161741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" event={"ID":"03c418d8-505d-4ea6-92da-18ae9bba17e5","Type":"ContainerStarted","Data":"94832dc67a3136c4323389bc872b8611b92f53984883b13193f7e99e0356e65f"}
Jul 22 15:13:39.161962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.161943    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:39.204236 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.204178    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj" podStartSLOduration=7.881278779 podStartE2EDuration="28.204162169s" totalImagesPullingTime="20.32288339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:39.203818298 +0000 UTC m=+28.645901744" watchObservedRunningTime="2026-07-22 15:13:39.204162169 +0000 UTC m=+28.646245616"
Jul 22 15:13:39.221764 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:39.221705    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdd5p" podStartSLOduration=3.018772732 podStartE2EDuration="28.221687457s" totalImagesPullingTime="25.202914725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:39.221663329 +0000 UTC m=+28.663746777" watchObservedRunningTime="2026-07-22 15:13:39.221687457 +0000 UTC m=+28.663770898"
Jul 22 15:13:40.164535 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:40.164442    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:40.165101 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:40.164545    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:40.180387 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:40.180360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:40.180592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:40.180579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:13:41.016109 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:41.016066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:41.016351 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:41.016152    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:41.016351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:41.016173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:41.016351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:41.016206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:41.016351 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:41.016273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:41.016351 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:41.016338    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:41.168434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:41.168400    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7f3a35-79f8-491e-82d2-f4cd36f14c4a" containerID="241cae778a77dbc0218aa1cdb85f012d5f74262c5de678e1d7d44ace4257c9b7" exitCode=0
Jul 22 15:13:41.168927 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:41.168502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerDied","Data":"241cae778a77dbc0218aa1cdb85f012d5f74262c5de678e1d7d44ace4257c9b7"}
Jul 22 15:13:43.015767 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:43.015737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:43.016163 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:43.015766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:43.016163 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:43.015766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:43.016163 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:43.015873    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:43.016163 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:43.015937    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:43.016163 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:43.016024    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:43.175326 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:43.175295    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7f3a35-79f8-491e-82d2-f4cd36f14c4a" containerID="221bb3b9dd1ec56224ff6f4bf9538cc7cba9313567c403d7b4a5233dcc3133af" exitCode=0
Jul 22 15:13:43.175523 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:43.175348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerDied","Data":"221bb3b9dd1ec56224ff6f4bf9538cc7cba9313567c403d7b4a5233dcc3133af"}
Jul 22 15:13:44.667369 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:44.667325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:44.667896 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.667514    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:44.667896 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.667598    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs podName:c259c676-391f-4797-a0f6-95f7e56b83b6 nodeName:}" failed. No retries permitted until 2026-07-22 15:14:16.667576078 +0000 UTC m=+66.109659516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs") pod "network-metrics-daemon-58d6n" (UID: "c259c676-391f-4797-a0f6-95f7e56b83b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:13:44.768591 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:44.768524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:44.768770 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.768678    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:44.768770 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.768746    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret podName:5b2ea94f-f08b-41e3-bb42-66b05a6f8e15 nodeName:}" failed. No retries permitted until 2026-07-22 15:14:16.768726693 +0000 UTC m=+66.210810130 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret") pod "global-pull-secret-syncer-4lwsx" (UID: "5b2ea94f-f08b-41e3-bb42-66b05a6f8e15") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:13:44.869260 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:44.869221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:44.869424 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.869372    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:13:44.869424 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.869390    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:13:44.869424 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.869405    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bdzcz for pod openshift-network-diagnostics/network-check-target-28qwf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:44.869574 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:44.869468    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz podName:b16d0e5c-2b37-48f1-9641-7612e14b2b12 nodeName:}" failed. No retries permitted until 2026-07-22 15:14:16.869448984 +0000 UTC m=+66.311532414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdzcz" (UniqueName: "kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz") pod "network-check-target-28qwf" (UID: "b16d0e5c-2b37-48f1-9641-7612e14b2b12") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:13:45.015990 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:45.015947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:45.015990 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:45.015976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:45.015990 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:45.015996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:45.016290 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:45.016123    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:45.016290 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:45.016187    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:45.016290 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:45.016267    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:47.016123 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:47.016086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:47.016542 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:47.016096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:47.016542 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:47.016196    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:47.016542 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:47.016228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:47.016542 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:47.016321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:47.016542 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:47.016380    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:48.191535 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:48.191434    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7f3a35-79f8-491e-82d2-f4cd36f14c4a" containerID="96571486327ee54d2680bf2c562e492c246d92f0b725b928e32c59e64ebd0b5b" exitCode=0
Jul 22 15:13:48.191535 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:48.191475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerDied","Data":"96571486327ee54d2680bf2c562e492c246d92f0b725b928e32c59e64ebd0b5b"}
Jul 22 15:13:49.015665 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:49.015631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:49.015886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:49.015633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:49.015886 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:49.015737    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:49.015886 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:49.015811    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:49.015886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:49.015646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:49.015886 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:49.015875    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:49.197055 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:49.197018    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7f3a35-79f8-491e-82d2-f4cd36f14c4a" containerID="7db6c6fea584c85b936e614de70abd877a52f595a227cda3b668fbe149d86368" exitCode=0
Jul 22 15:13:49.197446 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:49.197065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerDied","Data":"7db6c6fea584c85b936e614de70abd877a52f595a227cda3b668fbe149d86368"}
Jul 22 15:13:50.202275 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:50.202239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" event={"ID":"0f7f3a35-79f8-491e-82d2-f4cd36f14c4a","Type":"ContainerStarted","Data":"263f0aac4039565290918be25392cf5b9ff0336d433e653afd042e46ad7a0256"}
Jul 22 15:13:50.226016 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:50.225959    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c9vpv" podStartSLOduration=7.218684256 podStartE2EDuration="39.225943321s" totalImagesPullingTime="32.007259065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:50.225300678 +0000 UTC m=+39.667384124" watchObservedRunningTime="2026-07-22 15:13:50.225943321 +0000 UTC m=+39.668026767"
Jul 22 15:13:51.015723 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:51.015683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:51.015903 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:51.015772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:51.015903 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:51.015774    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:51.015903 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:51.015882    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:51.016044 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:51.015906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:51.016081 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:51.016039    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:53.015533 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:53.015317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:53.015994 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:53.015317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:53.015994 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:53.015626    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:53.015994 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:53.015334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:53.015994 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:53.015687    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:53.015994 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:53.015751    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:55.015141 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:55.015099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:55.015563 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:55.015099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:55.015563 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:55.015248    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:55.015563 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:55.015110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:55.015563 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:55.015328    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:55.015563 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:55.015383    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:57.015802 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.015754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:57.016264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.015776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:57.016264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.015785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:57.016264 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:57.015921    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:57.016264 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:57.016006    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:57.016264 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:57.016089    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:57.948956 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.948917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lwsx"]
Jul 22 15:13:57.949147 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.949051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:57.949210 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:57.949171    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:13:57.952532 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.952466    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-58d6n"]
Jul 22 15:13:57.952706 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.952607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:13:57.952758 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:57.952735    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:13:57.953305 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.953274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-28qwf"]
Jul 22 15:13:57.953420 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:57.953359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:13:57.953477 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:57.953456    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:13:59.015302 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:13:59.015268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:13:59.016061 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:13:59.015402    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:14:00.015227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:00.015191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:14:00.015424 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:00.015206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:14:00.015424 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:00.015321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-58d6n" podUID="c259c676-391f-4797-a0f6-95f7e56b83b6"
Jul 22 15:14:00.015424 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:00.015355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28qwf" podUID="b16d0e5c-2b37-48f1-9641-7612e14b2b12"
Jul 22 15:14:01.017846 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.017811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:14:01.018238 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:01.017925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lwsx" podUID="5b2ea94f-f08b-41e3-bb42-66b05a6f8e15"
Jul 22 15:14:01.428382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.428300    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-231.ec2.internal" event="NodeReady"
Jul 22 15:14:01.428557 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.428441    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 15:14:01.478784 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.478739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k4td6"]
Jul 22 15:14:01.501315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.501268    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jbrhr"]
Jul 22 15:14:01.501520 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.501427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.504997 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.504964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j8ghg\""
Jul 22 15:14:01.505153 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.504998    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 15:14:01.505153 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.504998    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 15:14:01.515239 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.515211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jbrhr"]
Jul 22 15:14:01.515239 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.515245    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k4td6"]
Jul 22 15:14:01.515404 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.515330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.524637 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.524611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zx2fs\""
Jul 22 15:14:01.525052 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.525032    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 15:14:01.525170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.525133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 15:14:01.525884 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.525867    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 15:14:01.584030 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.583988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0fe791c4-6fd8-438b-8df7-77722e6eff8a-config-volume\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.584030 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.584027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fe791c4-6fd8-438b-8df7-77722e6eff8a-metrics-tls\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.584249 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.584045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pck7r\" (UniqueName: \"kubernetes.io/projected/0fe791c4-6fd8-438b-8df7-77722e6eff8a-kube-api-access-pck7r\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.584249 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.584135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe791c4-6fd8-438b-8df7-77722e6eff8a-tmp-dir\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.685351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03051a9-1010-478e-977a-d453bcfa8068-cert\") pod \"ingress-canary-jbrhr\" (UID: \"e03051a9-1010-478e-977a-d453bcfa8068\") " pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.685351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0fe791c4-6fd8-438b-8df7-77722e6eff8a-config-volume\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.685351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fe791c4-6fd8-438b-8df7-77722e6eff8a-metrics-tls\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.685594 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pck7r\" (UniqueName: \"kubernetes.io/projected/0fe791c4-6fd8-438b-8df7-77722e6eff8a-kube-api-access-pck7r\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.685594 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5ctn\" (UniqueName: \"kubernetes.io/projected/e03051a9-1010-478e-977a-d453bcfa8068-kube-api-access-b5ctn\") pod \"ingress-canary-jbrhr\" (UID: \"e03051a9-1010-478e-977a-d453bcfa8068\") " pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.685594 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe791c4-6fd8-438b-8df7-77722e6eff8a-tmp-dir\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.685876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe791c4-6fd8-438b-8df7-77722e6eff8a-tmp-dir\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.685876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.685873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0fe791c4-6fd8-438b-8df7-77722e6eff8a-config-volume\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.689847 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.689710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fe791c4-6fd8-438b-8df7-77722e6eff8a-metrics-tls\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.695123 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.695100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pck7r\" (UniqueName: \"kubernetes.io/projected/0fe791c4-6fd8-438b-8df7-77722e6eff8a-kube-api-access-pck7r\") pod \"dns-default-k4td6\" (UID: \"0fe791c4-6fd8-438b-8df7-77722e6eff8a\") " pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.786394 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.786352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03051a9-1010-478e-977a-d453bcfa8068-cert\") pod \"ingress-canary-jbrhr\" (UID: \"e03051a9-1010-478e-977a-d453bcfa8068\") " pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.786394 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.786400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5ctn\" (UniqueName: \"kubernetes.io/projected/e03051a9-1010-478e-977a-d453bcfa8068-kube-api-access-b5ctn\") pod \"ingress-canary-jbrhr\" (UID: \"e03051a9-1010-478e-977a-d453bcfa8068\") " pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.789055 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.789009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03051a9-1010-478e-977a-d453bcfa8068-cert\") pod \"ingress-canary-jbrhr\" (UID: \"e03051a9-1010-478e-977a-d453bcfa8068\") " pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.799189 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.799155    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5ctn\" (UniqueName: \"kubernetes.io/projected/e03051a9-1010-478e-977a-d453bcfa8068-kube-api-access-b5ctn\") pod \"ingress-canary-jbrhr\" (UID: \"e03051a9-1010-478e-977a-d453bcfa8068\") " pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:01.813178 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.813141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:01.823307 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:01.823269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jbrhr"
Jul 22 15:14:02.015646 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.015614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:14:02.015832 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.015660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:14:02.018609 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.018586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 15:14:02.019012 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.018629    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 15:14:02.019375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.019354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-glqml\""
Jul 22 15:14:02.019532 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.019475    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-97vtz\""
Jul 22 15:14:02.019532 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.019484    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 15:14:02.026889 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.026863    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jbrhr"]
Jul 22 15:14:02.029931 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.029908    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k4td6"]
Jul 22 15:14:02.031141 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:02.031116    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode03051a9_1010_478e_977a_d453bcfa8068.slice/crio-7320781c61bb2640aaed168460ce80c47728b08e424bfbd3c1a971498d34c97e WatchSource:0}: Error finding container 7320781c61bb2640aaed168460ce80c47728b08e424bfbd3c1a971498d34c97e: Status 404 returned error can't find the container with id 7320781c61bb2640aaed168460ce80c47728b08e424bfbd3c1a971498d34c97e
Jul 22 15:14:02.035846 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:02.035823    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe791c4_6fd8_438b_8df7_77722e6eff8a.slice/crio-78b39c922a3b90272e7745df679b06c0f7e79776ca4959bc355dfbe6f8bb28e4 WatchSource:0}: Error finding container 78b39c922a3b90272e7745df679b06c0f7e79776ca4959bc355dfbe6f8bb28e4: Status 404 returned error can't find the container with id 78b39c922a3b90272e7745df679b06c0f7e79776ca4959bc355dfbe6f8bb28e4
Jul 22 15:14:02.233151 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.233052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jbrhr" event={"ID":"e03051a9-1010-478e-977a-d453bcfa8068","Type":"ContainerStarted","Data":"7320781c61bb2640aaed168460ce80c47728b08e424bfbd3c1a971498d34c97e"}
Jul 22 15:14:02.234117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:02.234095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k4td6" event={"ID":"0fe791c4-6fd8-438b-8df7-77722e6eff8a","Type":"ContainerStarted","Data":"78b39c922a3b90272e7745df679b06c0f7e79776ca4959bc355dfbe6f8bb28e4"}
Jul 22 15:14:03.016041 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.016007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:14:03.018824 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.018801    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 15:14:03.296856 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.296760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-6s5m6"]
Jul 22 15:14:03.319864 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.319830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-6s5m6"]
Jul 22 15:14:03.320054 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.319977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:03.322881 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.322857    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-csd2d\""
Jul 22 15:14:03.323115 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.323098    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 15:14:03.323317 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.323300    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 15:14:03.499198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.499162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swgxj\" (UniqueName: \"kubernetes.io/projected/5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf-kube-api-access-swgxj\") pod \"downloads-5bc67dcd46-6s5m6\" (UID: \"5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf\") " pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:03.599889 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.599801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swgxj\" (UniqueName: \"kubernetes.io/projected/5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf-kube-api-access-swgxj\") pod \"downloads-5bc67dcd46-6s5m6\" (UID: \"5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf\") " pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:03.610286 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.610256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swgxj\" (UniqueName: \"kubernetes.io/projected/5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf-kube-api-access-swgxj\") pod \"downloads-5bc67dcd46-6s5m6\" (UID: \"5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf\") " pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:03.632319 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:03.632067    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:04.082451 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:04.082424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-6s5m6"]
Jul 22 15:14:04.086995 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:04.086968    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d113dfc_e0d3_425e_8f5b_f6a71a6e5aaf.slice/crio-23a5d30412b4d18fa1a33e5383b8ea6974ebfe28fb182c38724e1cb166a02355 WatchSource:0}: Error finding container 23a5d30412b4d18fa1a33e5383b8ea6974ebfe28fb182c38724e1cb166a02355: Status 404 returned error can't find the container with id 23a5d30412b4d18fa1a33e5383b8ea6974ebfe28fb182c38724e1cb166a02355
Jul 22 15:14:04.244066 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:04.243799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k4td6" event={"ID":"0fe791c4-6fd8-438b-8df7-77722e6eff8a","Type":"ContainerStarted","Data":"a24d02cac43d0cb6f1395a295f843ce0fe367b8246295130219532a7490bbfda"}
Jul 22 15:14:04.245221 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:04.245191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-6s5m6" event={"ID":"5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf","Type":"ContainerStarted","Data":"23a5d30412b4d18fa1a33e5383b8ea6974ebfe28fb182c38724e1cb166a02355"}
Jul 22 15:14:05.250357 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:05.250314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k4td6" event={"ID":"0fe791c4-6fd8-438b-8df7-77722e6eff8a","Type":"ContainerStarted","Data":"d6c73cd520fcab2ccecc8efb71be23003c14bddff9482af6f68f423a46978439"}
Jul 22 15:14:05.251102 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:05.250454    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:05.272850 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:05.272779    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k4td6" podStartSLOduration=2.227123349 podStartE2EDuration="4.272758325s" totalImagesPullingTime="2.045634976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:05.271512995 +0000 UTC m=+54.713596437" watchObservedRunningTime="2026-07-22 15:14:05.272758325 +0000 UTC m=+54.714841773"
Jul 22 15:14:06.201843 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.201801    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"]
Jul 22 15:14:06.217140 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.217110    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"]
Jul 22 15:14:06.217330 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.217246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:06.219812 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.219741    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 22 15:14:06.219812 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.219767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-z56ht\""
Jul 22 15:14:06.254929 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.254893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jbrhr" event={"ID":"e03051a9-1010-478e-977a-d453bcfa8068","Type":"ContainerStarted","Data":"c867f94bcafbd9cc1ea125d3cb9b8104c43e196214ad57b13c1b1f4c4fca516a"}
Jul 22 15:14:06.270207 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.270146    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jbrhr" podStartSLOduration=2.138427108 podStartE2EDuration="5.270128514s" totalImagesPullingTime="3.131701406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:06.269752942 +0000 UTC m=+55.711836390" watchObservedRunningTime="2026-07-22 15:14:06.270128514 +0000 UTC m=+55.712211962"
Jul 22 15:14:06.321203 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.321168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1609947f-7031-4245-a946-035973ba3efd-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h7h47\" (UID: \"1609947f-7031-4245-a946-035973ba3efd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:06.421831 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.421790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1609947f-7031-4245-a946-035973ba3efd-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h7h47\" (UID: \"1609947f-7031-4245-a946-035973ba3efd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:06.422049 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:06.421914    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 22 15:14:06.422049 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:06.421977    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1609947f-7031-4245-a946-035973ba3efd-tls-certificates podName:1609947f-7031-4245-a946-035973ba3efd nodeName:}" failed. No retries permitted until 2026-07-22 15:14:06.921958995 +0000 UTC m=+56.364042423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1609947f-7031-4245-a946-035973ba3efd-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-h7h47" (UID: "1609947f-7031-4245-a946-035973ba3efd") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 22 15:14:06.544105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.544066    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lxqx6"]
Jul 22 15:14:06.565062 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.565002    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lxqx6"]
Jul 22 15:14:06.565285 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.565093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.567788 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.567758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 15:14:06.567919 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.567784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 15:14:06.567919 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.567741    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 15:14:06.568020 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.567962    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rppjt\""
Jul 22 15:14:06.568144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.568132    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 15:14:06.724316 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.724273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8093da5-8487-48e9-94bf-ab78aefa884b-data-volume\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.724539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.724322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52rd5\" (UniqueName: \"kubernetes.io/projected/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-api-access-52rd5\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.724539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.724438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c8093da5-8487-48e9-94bf-ab78aefa884b-crio-socket\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.724751 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.724545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.724751 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.724574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.825757 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.825653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.825757 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.825696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.825757 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.825722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8093da5-8487-48e9-94bf-ab78aefa884b-data-volume\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.825757 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.825744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52rd5\" (UniqueName: \"kubernetes.io/projected/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-api-access-52rd5\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:06.825802    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:06.825828    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:06.825877    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:07.325857892 +0000 UTC m=+56.767941318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:06.825901    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:07.325892881 +0000 UTC m=+56.767976349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : configmap "kube-rbac-proxy" not found
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.826005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c8093da5-8487-48e9-94bf-ab78aefa884b-crio-socket\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.825804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c8093da5-8487-48e9-94bf-ab78aefa884b-crio-socket\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.826064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.826044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8093da5-8487-48e9-94bf-ab78aefa884b-data-volume\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.836256 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.836227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52rd5\" (UniqueName: \"kubernetes.io/projected/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-api-access-52rd5\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:06.926896 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.926860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1609947f-7031-4245-a946-035973ba3efd-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h7h47\" (UID: \"1609947f-7031-4245-a946-035973ba3efd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:06.929886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:06.929852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1609947f-7031-4245-a946-035973ba3efd-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h7h47\" (UID: \"1609947f-7031-4245-a946-035973ba3efd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:07.127115 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:07.127029    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:07.322286 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:07.322016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"]
Jul 22 15:14:07.329460 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:07.329431    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1609947f_7031_4245_a946_035973ba3efd.slice/crio-6857e827af38fe02233825f01afa7d321ea93e332890be5a855cfd7439ef7e0b WatchSource:0}: Error finding container 6857e827af38fe02233825f01afa7d321ea93e332890be5a855cfd7439ef7e0b: Status 404 returned error can't find the container with id 6857e827af38fe02233825f01afa7d321ea93e332890be5a855cfd7439ef7e0b
Jul 22 15:14:07.331521 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:07.330719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:07.331521 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:07.330760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:07.331521 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:07.330829    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 15:14:07.331521 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:07.330898    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:08.330875318 +0000 UTC m=+57.772958743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : configmap "kube-rbac-proxy" not found
Jul 22 15:14:07.331521 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:07.330908    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:07.331521 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:07.330958    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:08.3309431 +0000 UTC m=+57.773026539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:07.561953 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:07.561917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k4td6_0fe791c4-6fd8-438b-8df7-77722e6eff8a/dns/0.log"
Jul 22 15:14:07.724119 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:07.724089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k4td6_0fe791c4-6fd8-438b-8df7-77722e6eff8a/kube-rbac-proxy/0.log"
Jul 22 15:14:08.262874 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:08.262830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47" event={"ID":"1609947f-7031-4245-a946-035973ba3efd","Type":"ContainerStarted","Data":"6857e827af38fe02233825f01afa7d321ea93e332890be5a855cfd7439ef7e0b"}
Jul 22 15:14:08.338576 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:08.338538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:08.339053 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:08.338590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:08.339053 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:08.338682    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 15:14:08.339053 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:08.338768    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:10.338744735 +0000 UTC m=+59.780828162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : configmap "kube-rbac-proxy" not found
Jul 22 15:14:08.339053 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:08.338773    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:08.339053 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:08.338825    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:10.338812943 +0000 UTC m=+59.780896368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:08.523817 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:08.523722    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hrk9j_c471fe40-e8a7-4c30-bf41-f3bcbe56d275/dns-node-resolver/0.log"
Jul 22 15:14:09.267215 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:09.267175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47" event={"ID":"1609947f-7031-4245-a946-035973ba3efd","Type":"ContainerStarted","Data":"a447b45966cfa7cea1be9af814f638267c9e3bb164daa0f855f1635bb9a9624c"}
Jul 22 15:14:09.267457 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:09.267437    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:09.273170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:09.273141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47"
Jul 22 15:14:09.283397 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:09.283344    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h7h47" podStartSLOduration=1.758953808 podStartE2EDuration="3.283324226s" totalImagesPullingTime="1.524370418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:09.282390833 +0000 UTC m=+58.724474281" watchObservedRunningTime="2026-07-22 15:14:09.283324226 +0000 UTC m=+58.725407677"
Jul 22 15:14:09.323172 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:09.323147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7zmpm_1a043e63-87db-4da7-8fbb-0f13536fcc38/node-ca/0.log"
Jul 22 15:14:10.289612 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.289578    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"]
Jul 22 15:14:10.291689 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.291671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.294900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.294877    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 15:14:10.295033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.294876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 22 15:14:10.296190 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.296164    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 22 15:14:10.296315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.296193    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 15:14:10.296315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.296175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zrr5f\""
Jul 22 15:14:10.304640 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.304614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 15:14:10.323764 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.323733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"]
Jul 22 15:14:10.353070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.353044    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jbrhr_e03051a9-1010-478e-977a-d453bcfa8068/serve-healthcheck-canary/0.log"
Jul 22 15:14:10.355173 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.355147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:10.355311 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.355181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:10.355311 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:10.355269    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 15:14:10.355383 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:10.355313    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:10.355383 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:10.355338    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:14.355322023 +0000 UTC m=+63.797405448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : configmap "kube-rbac-proxy" not found
Jul 22 15:14:10.355383 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:10.355367    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls podName:c8093da5-8487-48e9-94bf-ab78aefa884b nodeName:}" failed. No retries permitted until 2026-07-22 15:14:14.355355946 +0000 UTC m=+63.797439373 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxqx6" (UID: "c8093da5-8487-48e9-94bf-ab78aefa884b") : secret "insights-runtime-extractor-tls" not found
Jul 22 15:14:10.455844 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.455808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f853648-4345-4232-b7c9-48d74976166c-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.456030 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.455863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f853648-4345-4232-b7c9-48d74976166c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.456030 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.455972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f853648-4345-4232-b7c9-48d74976166c-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.456122 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.456031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gz6l\" (UniqueName: \"kubernetes.io/projected/4f853648-4345-4232-b7c9-48d74976166c-kube-api-access-2gz6l\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.556657 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.556547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gz6l\" (UniqueName: \"kubernetes.io/projected/4f853648-4345-4232-b7c9-48d74976166c-kube-api-access-2gz6l\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.556657 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.556615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f853648-4345-4232-b7c9-48d74976166c-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.556879 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.556665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f853648-4345-4232-b7c9-48d74976166c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.556879 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.556730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f853648-4345-4232-b7c9-48d74976166c-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.557587 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.557555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f853648-4345-4232-b7c9-48d74976166c-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.560936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.560906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f853648-4345-4232-b7c9-48d74976166c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.561044 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.560911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f853648-4345-4232-b7c9-48d74976166c-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.567471 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.567440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gz6l\" (UniqueName: \"kubernetes.io/projected/4f853648-4345-4232-b7c9-48d74976166c-kube-api-access-2gz6l\") pod \"prometheus-operator-5945b5cfd-cccgp\" (UID: \"4f853648-4345-4232-b7c9-48d74976166c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.601177 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.601134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"
Jul 22 15:14:10.740274 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:10.740244    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-cccgp"]
Jul 22 15:14:10.744103 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:10.744059    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f853648_4345_4232_b7c9_48d74976166c.slice/crio-f0f46d3fb9e207e3d135f9ea69a505593dd7f304df4d60fbc9caea5157f1b7d2 WatchSource:0}: Error finding container f0f46d3fb9e207e3d135f9ea69a505593dd7f304df4d60fbc9caea5157f1b7d2: Status 404 returned error can't find the container with id f0f46d3fb9e207e3d135f9ea69a505593dd7f304df4d60fbc9caea5157f1b7d2
Jul 22 15:14:11.272898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:11.272857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp" event={"ID":"4f853648-4345-4232-b7c9-48d74976166c","Type":"ContainerStarted","Data":"f0f46d3fb9e207e3d135f9ea69a505593dd7f304df4d60fbc9caea5157f1b7d2"}
Jul 22 15:14:12.184340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.184301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h82fj"
Jul 22 15:14:12.404718 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.404692    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d566bd748-r22x7"]
Jul 22 15:14:12.408714 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.408687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.412786 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.412753    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z9lvv\""
Jul 22 15:14:12.412918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.412841    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 22 15:14:12.412918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.412867    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 22 15:14:12.413018 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.412946    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 22 15:14:12.413130 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.413112    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 22 15:14:12.413193 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.413152    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 22 15:14:12.422083 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.422058    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d566bd748-r22x7"]
Jul 22 15:14:12.573898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.573862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-config\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.573898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.573902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-service-ca\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.574146 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.574001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-oauth-serving-cert\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.574146 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.574049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-oauth-config\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.574146 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.574116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-serving-cert\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.574312 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.574246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcs6v\" (UniqueName: \"kubernetes.io/projected/b00fc07c-1bd4-4855-be6f-8800aa41cb16-kube-api-access-pcs6v\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.675559 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.675517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-serving-cert\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.675743 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.675614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcs6v\" (UniqueName: \"kubernetes.io/projected/b00fc07c-1bd4-4855-be6f-8800aa41cb16-kube-api-access-pcs6v\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.675743 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.675645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-config\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.675743 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.675661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-service-ca\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.675743 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.675686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-oauth-serving-cert\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.675743 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.675707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-oauth-config\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.676407 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.676335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-service-ca\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.676407 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.676395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-oauth-serving-cert\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.676679 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.676626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-config\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.678286 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.678262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-serving-cert\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.678360 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.678344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-oauth-config\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.684980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.684955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcs6v\" (UniqueName: \"kubernetes.io/projected/b00fc07c-1bd4-4855-be6f-8800aa41cb16-kube-api-access-pcs6v\") pod \"console-7d566bd748-r22x7\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") " pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.721737 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.721643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:12.893400 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:12.893371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d566bd748-r22x7"]
Jul 22 15:14:13.281161 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:13.281121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d566bd748-r22x7" event={"ID":"b00fc07c-1bd4-4855-be6f-8800aa41cb16","Type":"ContainerStarted","Data":"04c8dcdca6a73a0611478a0bdb83c477d97d50f3d240e77417e178361f5ed18f"}
Jul 22 15:14:13.283394 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:13.283360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp" event={"ID":"4f853648-4345-4232-b7c9-48d74976166c","Type":"ContainerStarted","Data":"2268dbdac6825b56f471578b2c86e3cf50ded6e13e58a00a8d0bc4a86bda38ec"}
Jul 22 15:14:13.283570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:13.283420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp" event={"ID":"4f853648-4345-4232-b7c9-48d74976166c","Type":"ContainerStarted","Data":"fdd9a2ff230f25c168406b5ff6c826ccd3688c59b236c649ca9162360a6702c3"}
Jul 22 15:14:13.302192 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:13.302131    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cccgp" podStartSLOduration=1.700792105 podStartE2EDuration="3.302107164s" totalImagesPullingTime="1.601315059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:13.301805891 +0000 UTC m=+62.743889346" watchObservedRunningTime="2026-07-22 15:14:13.302107164 +0000 UTC m=+62.744190612"
Jul 22 15:14:14.389101 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.389060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:14.389630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.389112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:14.389849 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.389825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c8093da5-8487-48e9-94bf-ab78aefa884b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:14.395247 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.395183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c8093da5-8487-48e9-94bf-ab78aefa884b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxqx6\" (UID: \"c8093da5-8487-48e9-94bf-ab78aefa884b\") " pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:14.678997 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.678834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rppjt\""
Jul 22 15:14:14.687468 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.687432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lxqx6"
Jul 22 15:14:14.852468 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:14.852432    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lxqx6"]
Jul 22 15:14:14.856357 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:14.856326    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8093da5_8487_48e9_94bf_ab78aefa884b.slice/crio-9056ad404083b955f2a7f475892c57ad03773954b2b00d617d33a36b0b9a373e WatchSource:0}: Error finding container 9056ad404083b955f2a7f475892c57ad03773954b2b00d617d33a36b0b9a373e: Status 404 returned error can't find the container with id 9056ad404083b955f2a7f475892c57ad03773954b2b00d617d33a36b0b9a373e
Jul 22 15:14:15.257784 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.257750    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k4td6"
Jul 22 15:14:15.297360 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.295473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxqx6" event={"ID":"c8093da5-8487-48e9-94bf-ab78aefa884b","Type":"ContainerStarted","Data":"9056ad404083b955f2a7f475892c57ad03773954b2b00d617d33a36b0b9a373e"}
Jul 22 15:14:15.694608 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.694519    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"]
Jul 22 15:14:15.699103 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.699076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.702112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.701876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 22 15:14:15.702112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.701895    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 22 15:14:15.703469 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.703441    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pm7fk\""
Jul 22 15:14:15.704551 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.704276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.704551 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.704314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv6x4\" (UniqueName: \"kubernetes.io/projected/498d107e-12ee-435e-8ca0-5114e13f7e5c-kube-api-access-nv6x4\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.704551 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.704381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/498d107e-12ee-435e-8ca0-5114e13f7e5c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.704551 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.704420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.711995 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.711973    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"]
Jul 22 15:14:15.730266 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.730230    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"]
Jul 22 15:14:15.734390 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.734361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.737387 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.737358    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 22 15:14:15.737531 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.737466    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 22 15:14:15.737577 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.737523    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sc6hd\""
Jul 22 15:14:15.740204 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.738036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 22 15:14:15.748570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.748547    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2x9qs"]
Jul 22 15:14:15.752690 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.752663    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"]
Jul 22 15:14:15.752893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.752770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.755049 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.755030    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 15:14:15.755606 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.755300    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 15:14:15.755606 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.755469    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 15:14:15.755786 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.755624    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m4x9h\""
Jul 22 15:14:15.805071 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tsqr\" (UniqueName: \"kubernetes.io/projected/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-api-access-7tsqr\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.805264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs6gt\" (UniqueName: \"kubernetes.io/projected/1911e184-b292-4084-a23b-048b01b90de7-kube-api-access-cs6gt\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a02f7fd1-174d-47c2-aeaa-062a93041903-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.805264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-wtmp\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-root\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1911e184-b292-4084-a23b-048b01b90de7-metrics-client-ca\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-tls\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.805434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.805434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv6x4\" (UniqueName: \"kubernetes.io/projected/498d107e-12ee-435e-8ca0-5114e13f7e5c-kube-api-access-nv6x4\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-textfile\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:15.805516    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/498d107e-12ee-435e-8ca0-5114e13f7e5c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805556    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-sys\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:15.805587    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-tls podName:498d107e-12ee-435e-8ca0-5114e13f7e5c nodeName:}" failed. No retries permitted until 2026-07-22 15:14:16.305563243 +0000 UTC m=+65.747646670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-56v56" (UID: "498d107e-12ee-435e-8ca0-5114e13f7e5c") : secret "openshift-state-metrics-tls" not found
Jul 22 15:14:15.805681 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.805981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a02f7fd1-174d-47c2-aeaa-062a93041903-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.805981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.805981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-accelerators-collector-config\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.805981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.805825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.806257 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.806238    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/498d107e-12ee-435e-8ca0-5114e13f7e5c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.808674 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.808641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.819457 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.819424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv6x4\" (UniqueName: \"kubernetes.io/projected/498d107e-12ee-435e-8ca0-5114e13f7e5c-kube-api-access-nv6x4\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:15.906622 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.906622 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-sys\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.906870 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a02f7fd1-174d-47c2-aeaa-062a93041903-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.906870 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.906870 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-accelerators-collector-config\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.906870 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.906870 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-sys\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.906870 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tsqr\" (UniqueName: \"kubernetes.io/projected/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-api-access-7tsqr\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:15.906874    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:15.906954    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-tls podName:a02f7fd1-174d-47c2-aeaa-062a93041903 nodeName:}" failed. No retries permitted until 2026-07-22 15:14:16.406928712 +0000 UTC m=+65.849012137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-dtb6f" (UID: "a02f7fd1-174d-47c2-aeaa-062a93041903") : secret "kube-state-metrics-tls" not found
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.906983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6gt\" (UniqueName: \"kubernetes.io/projected/1911e184-b292-4084-a23b-048b01b90de7-kube-api-access-cs6gt\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a02f7fd1-174d-47c2-aeaa-062a93041903-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-wtmp\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-root\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1911e184-b292-4084-a23b-048b01b90de7-metrics-client-ca\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-tls\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.907426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-root\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-textfile\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a02f7fd1-174d-47c2-aeaa-062a93041903-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.907426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-wtmp\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907740 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a02f7fd1-174d-47c2-aeaa-062a93041903-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.907971 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1911e184-b292-4084-a23b-048b01b90de7-metrics-client-ca\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.907971 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.907952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.908097 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.908037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-accelerators-collector-config\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.908260 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.908236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-textfile\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.909688 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.909651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.910333 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.910309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:15.910880 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.910854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1911e184-b292-4084-a23b-048b01b90de7-node-exporter-tls\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.919128 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.919095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs6gt\" (UniqueName: \"kubernetes.io/projected/1911e184-b292-4084-a23b-048b01b90de7-kube-api-access-cs6gt\") pod \"node-exporter-2x9qs\" (UID: \"1911e184-b292-4084-a23b-048b01b90de7\") " pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:15.922279 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:15.922253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tsqr\" (UniqueName: \"kubernetes.io/projected/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-api-access-7tsqr\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:16.064097 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.064058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2x9qs"
Jul 22 15:14:16.312016 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.311975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:16.314785 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.314723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/498d107e-12ee-435e-8ca0-5114e13f7e5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-56v56\" (UID: \"498d107e-12ee-435e-8ca0-5114e13f7e5c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:16.409813 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.409779    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7998484d86-fhxs6"]
Jul 22 15:14:16.413434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.413399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:16.416705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.416478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.417462 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.417437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a02f7fd1-174d-47c2-aeaa-062a93041903-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-dtb6f\" (UID: \"a02f7fd1-174d-47c2-aeaa-062a93041903\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:16.426605 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.426578    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 22 15:14:16.427763 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.427737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7998484d86-fhxs6"]
Jul 22 15:14:16.452346 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:16.452304    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1911e184_b292_4084_a23b_048b01b90de7.slice/crio-e0f224d9692af2a3d0cd67698e60ee6349002ea5750860c5811539b211e56890 WatchSource:0}: Error finding container e0f224d9692af2a3d0cd67698e60ee6349002ea5750860c5811539b211e56890: Status 404 returned error can't find the container with id e0f224d9692af2a3d0cd67698e60ee6349002ea5750860c5811539b211e56890
Jul 22 15:14:16.514560 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-serving-cert\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.514560 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-trusted-ca-bundle\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.514747 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cft82\" (UniqueName: \"kubernetes.io/projected/8bf60db4-6230-4df0-a1f2-379a22fdc511-kube-api-access-cft82\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.514747 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-config\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.514747 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-oauth-serving-cert\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.514880 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-service-ca\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.514936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.514885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-oauth-config\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.613408 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.613375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"
Jul 22 15:14:16.615889 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.615856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-service-ca\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.615907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-oauth-config\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.615944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-serving-cert\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.615968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-trusted-ca-bundle\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.616005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cft82\" (UniqueName: \"kubernetes.io/projected/8bf60db4-6230-4df0-a1f2-379a22fdc511-kube-api-access-cft82\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.616042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-config\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.616101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-oauth-serving-cert\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.616602 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.616574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-service-ca\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.617081 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.616876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-oauth-serving-cert\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.617451 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.617408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-trusted-ca-bundle\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.617609 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.617569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-config\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.621333 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.621038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-oauth-config\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.621454 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.621432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-serving-cert\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.629332 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.629276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cft82\" (UniqueName: \"kubernetes.io/projected/8bf60db4-6230-4df0-a1f2-379a22fdc511-kube-api-access-cft82\") pod \"console-7998484d86-fhxs6\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") " pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.648966 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.648652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"
Jul 22 15:14:16.722620 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.722576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:14:16.725281 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.725029    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 15:14:16.730146 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.730120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:16.740191 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.740162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c259c676-391f-4797-a0f6-95f7e56b83b6-metrics-certs\") pod \"network-metrics-daemon-58d6n\" (UID: \"c259c676-391f-4797-a0f6-95f7e56b83b6\") " pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:14:16.787544 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.785536    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:14:16.792684 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.790860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.801846 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.794174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rjq9p\""
Jul 22 15:14:16.801846 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.794570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 15:14:16.801846 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.794813    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 15:14:16.801846 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.794990    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 15:14:16.805375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.803616    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 15:14:16.805375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.803772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 15:14:16.805375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.804277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 15:14:16.805375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.804720    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 15:14:16.805375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.804979    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 15:14:16.805375 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.805358    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 15:14:16.812322 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.812294    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:14:16.825288 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.825196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.825577 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.825552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.825971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-web-config\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-tls-assets\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkm9p\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-kube-api-access-hkm9p\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-config-out\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.826580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.826359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-config-volume\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.831593 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.830988    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 15:14:16.857559 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.854446    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56"]
Jul 22 15:14:16.859803 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.858749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b2ea94f-f08b-41e3-bb42-66b05a6f8e15-original-pull-secret\") pod \"global-pull-secret-syncer-4lwsx\" (UID: \"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15\") " pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:14:16.884787 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.884126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-dtb6f"]
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-tls-assets\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkm9p\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-kube-api-access-hkm9p\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-config-out\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.927509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.927470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-config-volume\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-web-config\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.928508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.928255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.929833 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.929439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.932479 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.931101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.932479 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.931843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.936220 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.936201    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 15:14:16.943111 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.941441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.943111 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.941595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.943320 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.943295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-web-config\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.943519 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.943468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.945898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.945424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-config-volume\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.945898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.945581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkm9p\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-kube-api-access-hkm9p\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.945898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.945791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-config-out\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.945898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.945844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.946218 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.946034    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 15:14:16.946607 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.946565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-tls-assets\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.947038 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.946999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:16.957677 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.957654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdzcz\" (UniqueName: \"kubernetes.io/projected/b16d0e5c-2b37-48f1-9641-7612e14b2b12-kube-api-access-bdzcz\") pod \"network-check-target-28qwf\" (UID: \"b16d0e5c-2b37-48f1-9641-7612e14b2b12\") " pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:14:16.964238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:16.964208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7998484d86-fhxs6"]
Jul 22 15:14:17.029392 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.029362    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-glqml\""
Jul 22 15:14:17.035143 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.035104    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-97vtz\""
Jul 22 15:14:17.036521 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.036467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:14:17.042866 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.042837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-58d6n"
Jul 22 15:14:17.130106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.129844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lwsx"
Jul 22 15:14:17.182383 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.182109    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:14:17.260435 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.260396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-28qwf"]
Jul 22 15:14:17.267945 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:17.267885    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16d0e5c_2b37_48f1_9641_7612e14b2b12.slice/crio-d347a31c6749f37f877afcb87294c0f6ff398254fac7a0101ad07b3eca346428 WatchSource:0}: Error finding container d347a31c6749f37f877afcb87294c0f6ff398254fac7a0101ad07b3eca346428: Status 404 returned error can't find the container with id d347a31c6749f37f877afcb87294c0f6ff398254fac7a0101ad07b3eca346428
Jul 22 15:14:17.309404 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.309351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-58d6n"]
Jul 22 15:14:17.321056 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.318754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7998484d86-fhxs6" event={"ID":"8bf60db4-6230-4df0-a1f2-379a22fdc511","Type":"ContainerStarted","Data":"9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b"}
Jul 22 15:14:17.321056 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.321007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7998484d86-fhxs6" event={"ID":"8bf60db4-6230-4df0-a1f2-379a22fdc511","Type":"ContainerStarted","Data":"afc4d23e56fa3390c088200128d876d2e7365849a6481389f53ee4378cd17ff8"}
Jul 22 15:14:17.329601 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.329325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2x9qs" event={"ID":"1911e184-b292-4084-a23b-048b01b90de7","Type":"ContainerStarted","Data":"e0f224d9692af2a3d0cd67698e60ee6349002ea5750860c5811539b211e56890"}
Jul 22 15:14:17.344409 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.344373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d566bd748-r22x7" event={"ID":"b00fc07c-1bd4-4855-be6f-8800aa41cb16","Type":"ContainerStarted","Data":"505883ac247892aedf3a60d17d400716bec79e5084b9f912cec8ed6db900c43b"}
Jul 22 15:14:17.349093 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.348038    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7998484d86-fhxs6" podStartSLOduration=1.348019053 podStartE2EDuration="1.348019053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:17.347871661 +0000 UTC m=+66.789955111" watchObservedRunningTime="2026-07-22 15:14:17.348019053 +0000 UTC m=+66.790102502"
Jul 22 15:14:17.350523 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.350426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-28qwf" event={"ID":"b16d0e5c-2b37-48f1-9641-7612e14b2b12","Type":"ContainerStarted","Data":"d347a31c6749f37f877afcb87294c0f6ff398254fac7a0101ad07b3eca346428"}
Jul 22 15:14:17.358084 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.355390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f" event={"ID":"a02f7fd1-174d-47c2-aeaa-062a93041903","Type":"ContainerStarted","Data":"eb078db32da77c49b08d61d8c1bc4119b10a8f57358fe4419b05cb9cb769305b"}
Jul 22 15:14:17.366923 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.366766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56" event={"ID":"498d107e-12ee-435e-8ca0-5114e13f7e5c","Type":"ContainerStarted","Data":"b7c6c000009b0fa02c6448b616df4fd22eeb0a3fa2d9216393f6d98e4b8d619d"}
Jul 22 15:14:17.366923 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.366843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56" event={"ID":"498d107e-12ee-435e-8ca0-5114e13f7e5c","Type":"ContainerStarted","Data":"136ae51b440b146b0a600e406ae373899e9224678ebd22c3e3539bbb34ba65b6"}
Jul 22 15:14:17.366923 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.366874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56" event={"ID":"498d107e-12ee-435e-8ca0-5114e13f7e5c","Type":"ContainerStarted","Data":"a41b0379b7692d9a3ab7a83ba129fdfb40690cb7e6e41ec7a8c5f74d378f7a6b"}
Jul 22 15:14:17.370725 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.370652    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d566bd748-r22x7" podStartSLOduration=1.7704659710000001 podStartE2EDuration="5.370614494s" totalImagesPullingTime="3.600148523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:17.369953331 +0000 UTC m=+66.812036778" watchObservedRunningTime="2026-07-22 15:14:17.370614494 +0000 UTC m=+66.812697940"
Jul 22 15:14:17.387038 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.386958    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lwsx"]
Jul 22 15:14:17.451464 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.451352    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:14:17.456660 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:17.456625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d215ec_ddda_48d6_b726_f9165f3c5299.slice/crio-5a5353b3d7c0bfd3e818e4404ed9143f1ae02ed0be82a595648c7ce500e3b05c WatchSource:0}: Error finding container 5a5353b3d7c0bfd3e818e4404ed9143f1ae02ed0be82a595648c7ce500e3b05c: Status 404 returned error can't find the container with id 5a5353b3d7c0bfd3e818e4404ed9143f1ae02ed0be82a595648c7ce500e3b05c
Jul 22 15:14:17.715995 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.714146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-69b988cc46-6fzsm"]
Jul 22 15:14:17.719552 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.718367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.724075 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.722902    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 22 15:14:17.724075 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.723174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 22 15:14:17.724075 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.723351    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-x7p2t\""
Jul 22 15:14:17.724075 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.723539    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 22 15:14:17.724075 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.723721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c5rcel9vrru52\""
Jul 22 15:14:17.724075 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.723889    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 22 15:14:17.731067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.729916    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.735816    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69b988cc46-6fzsm"]
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.745359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.745441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.745518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-grpc-tls\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.745578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-metrics-client-ca\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.746275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmd5g\" (UniqueName: \"kubernetes.io/projected/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-kube-api-access-rmd5g\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.746339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-tls\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.746414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.746695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.746463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.848234 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.847638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmd5g\" (UniqueName: \"kubernetes.io/projected/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-kube-api-access-rmd5g\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.848234 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.847753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-tls\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.848234 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.847818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.848234 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.847866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.849660 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.848648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.849660 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.848719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.849660 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.848748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-grpc-tls\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.849660 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.848801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-metrics-client-ca\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.849660 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.849611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-metrics-client-ca\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.858217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.856482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-grpc-tls\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.858217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.857018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.858217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.858099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.859815 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.858938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.859815 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.859273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.861319 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.861275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-secret-thanos-querier-tls\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:17.865229 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:17.865179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmd5g\" (UniqueName: \"kubernetes.io/projected/a3e3074a-2f4f-47e5-99e9-a5646ea37d05-kube-api-access-rmd5g\") pod \"thanos-querier-69b988cc46-6fzsm\" (UID: \"a3e3074a-2f4f-47e5-99e9-a5646ea37d05\") " pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:18.056619 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.056320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:18.293262 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.293146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69b988cc46-6fzsm"]
Jul 22 15:14:18.301314 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:18.301269    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e3074a_2f4f_47e5_99e9_a5646ea37d05.slice/crio-57b8247131fba40f4aaa8f1d9761c08b99d1ae74c8c99e95353bef137e681f9f WatchSource:0}: Error finding container 57b8247131fba40f4aaa8f1d9761c08b99d1ae74c8c99e95353bef137e681f9f: Status 404 returned error can't find the container with id 57b8247131fba40f4aaa8f1d9761c08b99d1ae74c8c99e95353bef137e681f9f
Jul 22 15:14:18.404315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.404274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"5a5353b3d7c0bfd3e818e4404ed9143f1ae02ed0be82a595648c7ce500e3b05c"}
Jul 22 15:14:18.409847 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.409781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lwsx" event={"ID":"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15","Type":"ContainerStarted","Data":"fe2a6d8d10684b3c50d1d188db73ad2ef5bd6744e61cba8ce3bc31c63bdcf78c"}
Jul 22 15:14:18.416397 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.416358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-58d6n" event={"ID":"c259c676-391f-4797-a0f6-95f7e56b83b6","Type":"ContainerStarted","Data":"a0cca183eb8ccd17872728ec65c84e8dc983e2072a68dc4eddb74352f645da85"}
Jul 22 15:14:18.419087 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.419056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2x9qs" event={"ID":"1911e184-b292-4084-a23b-048b01b90de7","Type":"ContainerStarted","Data":"0017abd055cdbfc2d585f071174023e8a6c248aad3ddea16163068b1a39d6dca"}
Jul 22 15:14:18.425184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:18.424676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"57b8247131fba40f4aaa8f1d9761c08b99d1ae74c8c99e95353bef137e681f9f"}
Jul 22 15:14:19.432941 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:19.432786    2580 generic.go:356] "Generic (PLEG): container finished" podID="1911e184-b292-4084-a23b-048b01b90de7" containerID="0017abd055cdbfc2d585f071174023e8a6c248aad3ddea16163068b1a39d6dca" exitCode=0
Jul 22 15:14:19.432941 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:19.432877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2x9qs" event={"ID":"1911e184-b292-4084-a23b-048b01b90de7","Type":"ContainerDied","Data":"0017abd055cdbfc2d585f071174023e8a6c248aad3ddea16163068b1a39d6dca"}
Jul 22 15:14:20.012455 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.012326    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f789fd5c9-zfch9"]
Jul 22 15:14:20.018350 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.018216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.022191 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.022150    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 22 15:14:20.023590 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.023543    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 22 15:14:20.026900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.023946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 15:14:20.026900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.024164    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-h2vg8\""
Jul 22 15:14:20.026900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.024348    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d6g1efmu2auu9\""
Jul 22 15:14:20.026900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.024575    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 22 15:14:20.026900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.026825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f789fd5c9-zfch9"]
Jul 22 15:14:20.074836 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.074791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-secret-metrics-server-client-certs\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.074836 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.074841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ae7a5864-df9b-4d91-b451-5644b081226c-metrics-server-audit-profiles\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.075090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.074946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ae7a5864-df9b-4d91-b451-5644b081226c-audit-log\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.075090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.074994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-secret-metrics-server-tls\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.075090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.075020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-client-ca-bundle\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.075090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.075047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae7a5864-df9b-4d91-b451-5644b081226c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.075291 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.075107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pcc2\" (UniqueName: \"kubernetes.io/projected/ae7a5864-df9b-4d91-b451-5644b081226c-kube-api-access-5pcc2\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-secret-metrics-server-client-certs\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ae7a5864-df9b-4d91-b451-5644b081226c-metrics-server-audit-profiles\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ae7a5864-df9b-4d91-b451-5644b081226c-audit-log\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-secret-metrics-server-tls\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-client-ca-bundle\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae7a5864-df9b-4d91-b451-5644b081226c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.177310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.176425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pcc2\" (UniqueName: \"kubernetes.io/projected/ae7a5864-df9b-4d91-b451-5644b081226c-kube-api-access-5pcc2\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.180416 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.180196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae7a5864-df9b-4d91-b451-5644b081226c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.180892 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.180806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ae7a5864-df9b-4d91-b451-5644b081226c-audit-log\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.183367 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.183310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ae7a5864-df9b-4d91-b451-5644b081226c-metrics-server-audit-profiles\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.188110 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.187903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-secret-metrics-server-client-certs\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.188110 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.188059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-secret-metrics-server-tls\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.191950 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.191872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pcc2\" (UniqueName: \"kubernetes.io/projected/ae7a5864-df9b-4d91-b451-5644b081226c-kube-api-access-5pcc2\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.195327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.195255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae7a5864-df9b-4d91-b451-5644b081226c-client-ca-bundle\") pod \"metrics-server-7f789fd5c9-zfch9\" (UID: \"ae7a5864-df9b-4d91-b451-5644b081226c\") " pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.339810 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.339655    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:14:20.463837 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.463793    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"]
Jul 22 15:14:20.483959 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.483915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"]
Jul 22 15:14:20.484142 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.484067    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:20.486884 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.486694    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xlplg\""
Jul 22 15:14:20.486884 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.486703    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 22 15:14:20.580522 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.580445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wfmm\" (UniqueName: \"kubernetes.io/projected/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-kube-api-access-7wfmm\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:20.580831 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.580656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:20.681941 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.681845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:20.682097 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.681921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wfmm\" (UniqueName: \"kubernetes.io/projected/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-kube-api-access-7wfmm\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:20.682097 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:20.682028    2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 22 15:14:20.682206 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:20.682120    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-monitoring-plugin-cert podName:b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2 nodeName:}" failed. No retries permitted until 2026-07-22 15:14:21.182097196 +0000 UTC m=+70.624180628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-qdnvz" (UID: "b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2") : secret "monitoring-plugin-cert" not found
Jul 22 15:14:20.701767 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:20.701563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wfmm\" (UniqueName: \"kubernetes.io/projected/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-kube-api-access-7wfmm\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:21.186304 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.186258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:21.189213 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.189188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qdnvz\" (UID: \"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:21.396017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.395979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:21.968681 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.968619    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:14:21.971837 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.971771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:21.975212 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.975154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 15:14:21.975212 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.975178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 15:14:21.975735 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.975708    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 15:14:21.975859 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.975752    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5kfpuudmhq6r\""
Jul 22 15:14:21.975966 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.975950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 15:14:21.976053 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.976035    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kxk4d\""
Jul 22 15:14:21.976159 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.976144    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 15:14:21.976237 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.976145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 15:14:21.976340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.976324    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 15:14:21.976917 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.976884    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 15:14:21.977160 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.977013    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 15:14:21.977578 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.977357    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 15:14:21.977578 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.977417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 15:14:21.977929 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.977911    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 15:14:21.980760 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.980571    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 15:14:21.989697 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.989666    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 15:14:21.992664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:21.992542    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:14:22.095102 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095303 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095303 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095303 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095470 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095470 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095470 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095470 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs5l7\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-kube-api-access-bs5l7\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-config\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.095868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.096126 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.095892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197472 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197472 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197808 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197808 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197808 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197949 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197949 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197949 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs5l7\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-kube-api-access-bs5l7\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197949 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.197949 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.197997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-config\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198482 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198482 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.198482 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.198932    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.199276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.200202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.200569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.200584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.200966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.201233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.202699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.202537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.203550 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.203474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.203820 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.203764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.204687 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.204660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-config\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.206817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.206897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.207091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.207275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.207524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.207733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.208154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.208103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.210711 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.210655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.211865 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.211844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs5l7\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-kube-api-access-bs5l7\") pod \"prometheus-k8s-0\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.285581 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.285539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:22.722522 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.722407    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:22.722714 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.722540    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:22.727982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:22.727952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:23.201504 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:23.201448    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d566bd748-r22x7"]
Jul 22 15:14:23.454593 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:23.454513    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:26.731079 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:26.731041    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:26.731555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:26.731136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:26.736036 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:26.736015    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:27.472184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:27.472153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:33.738729 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:33.738579    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7998484d86-fhxs6"]
Jul 22 15:14:35.232910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:35.232871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"]
Jul 22 15:14:35.464660 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:35.464623    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb142f82e_ecc3_47bd_9fa2_9c7e4c9c6ff2.slice/crio-01bc9a6fba12b6d6db530f40e880bda1633364e2812d26369d4d746c71458401 WatchSource:0}: Error finding container 01bc9a6fba12b6d6db530f40e880bda1633364e2812d26369d4d746c71458401: Status 404 returned error can't find the container with id 01bc9a6fba12b6d6db530f40e880bda1633364e2812d26369d4d746c71458401
Jul 22 15:14:35.504999 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:35.504931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz" event={"ID":"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2","Type":"ContainerStarted","Data":"01bc9a6fba12b6d6db530f40e880bda1633364e2812d26369d4d746c71458401"}
Jul 22 15:14:35.673908 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:35.665032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:14:35.703943 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:35.703913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f789fd5c9-zfch9"]
Jul 22 15:14:35.751778 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:14:35.751735    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7a5864_df9b_4d91_b451_5644b081226c.slice/crio-cfff0fa9be55dcbb230f22df5236383c810caf3d25b0e3cf6a13ce4e6518d4f2 WatchSource:0}: Error finding container cfff0fa9be55dcbb230f22df5236383c810caf3d25b0e3cf6a13ce4e6518d4f2: Status 404 returned error can't find the container with id cfff0fa9be55dcbb230f22df5236383c810caf3d25b0e3cf6a13ce4e6518d4f2
Jul 22 15:14:36.520448 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.520324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxqx6" event={"ID":"c8093da5-8487-48e9-94bf-ab78aefa884b","Type":"ContainerStarted","Data":"cbe5d22432276c308b647d7ea3868789515a64e12db8b2384c0d8d7a46d1bf7c"}
Jul 22 15:14:36.526377 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.525244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f" event={"ID":"a02f7fd1-174d-47c2-aeaa-062a93041903","Type":"ContainerStarted","Data":"a3294602d9a1c621c5bc8f900ba80340899b251da835d98df982aee21264528b"}
Jul 22 15:14:36.526377 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.525319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f" event={"ID":"a02f7fd1-174d-47c2-aeaa-062a93041903","Type":"ContainerStarted","Data":"df5d616406376fd72871e2ca726bcc4f059b69d0e157cd208cca7eb98fffe8be"}
Jul 22 15:14:36.526377 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.525350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f" event={"ID":"a02f7fd1-174d-47c2-aeaa-062a93041903","Type":"ContainerStarted","Data":"42538b558c3fa3d98dba0d08b50e04bb2c400c735898837353f514e987325de6"}
Jul 22 15:14:36.532570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.531998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56" event={"ID":"498d107e-12ee-435e-8ca0-5114e13f7e5c","Type":"ContainerStarted","Data":"92a62cab1ad23b936982a254fc0d7a1eae8c03530c9fa5b255da8bb1a75e1e94"}
Jul 22 15:14:36.535551 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.535191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lwsx" event={"ID":"5b2ea94f-f08b-41e3-bb42-66b05a6f8e15","Type":"ContainerStarted","Data":"898559899f232bb4372dd060af28f5de710da9cd803ef047f2c850bf7021c32f"}
Jul 22 15:14:36.538674 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.538563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-58d6n" event={"ID":"c259c676-391f-4797-a0f6-95f7e56b83b6","Type":"ContainerStarted","Data":"56b315b610142bcb62e5fdeeebd59c62093ee7f563626a4fefe469172a4bc389"}
Jul 22 15:14:36.539578 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.538884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-58d6n" event={"ID":"c259c676-391f-4797-a0f6-95f7e56b83b6","Type":"ContainerStarted","Data":"e1ad928adb878649f85e9074e8d99d6a0488bf8cbc736211f39dcfedf5c5a2b1"}
Jul 22 15:14:36.543090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.543025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9" event={"ID":"ae7a5864-df9b-4d91-b451-5644b081226c","Type":"ContainerStarted","Data":"cfff0fa9be55dcbb230f22df5236383c810caf3d25b0e3cf6a13ce4e6518d4f2"}
Jul 22 15:14:36.548540 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.548425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"}
Jul 22 15:14:36.549885 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.549699    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8" exitCode=0
Jul 22 15:14:36.549885 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.549810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"66096668fae8b048b8490fd021800864056e9285ceeaabcc8e4171f892f5e4b4"}
Jul 22 15:14:36.553933 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.553357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-6s5m6" event={"ID":"5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf","Type":"ContainerStarted","Data":"2397515ce1eaa46890d1fc7e08398678fe26dc337e71a57acd926c92527d1296"}
Jul 22 15:14:36.553933 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.553901    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:36.558749 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.558310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2x9qs" event={"ID":"1911e184-b292-4084-a23b-048b01b90de7","Type":"ContainerStarted","Data":"022aa4e2b5c46881432336c4ee8bf30c61bbc1152e4da0f609cd8c4366b8ce24"}
Jul 22 15:14:36.558749 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.558376    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2x9qs" event={"ID":"1911e184-b292-4084-a23b-048b01b90de7","Type":"ContainerStarted","Data":"f539998aea5ba4eddd48d8a96339b64bcddeb2c8ab95bd35c58ab45ff0d74161"}
Jul 22 15:14:36.560420 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.560392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-6s5m6"
Jul 22 15:14:36.564026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.563827    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"1d005031e74ceb3158f9244f717959a0b2549b57d631f5cb3c154d9299231f8c"}
Jul 22 15:14:36.564026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.563911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"0c113213693034e9f4b7b91b169db785178632828fda59dc3eb3e25425e44e8c"}
Jul 22 15:14:36.564026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.563951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"7e4422f3535d159751cb4d5918797e0eda9e2e5f80891e6a33add716dcf6d7d0"}
Jul 22 15:14:36.568965 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.568535    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-dtb6f" podStartSLOduration=3.032822079 podStartE2EDuration="21.568511166s" totalImagesPullingTime="18.535689087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.547151465 +0000 UTC m=+85.989234914" watchObservedRunningTime="2026-07-22 15:14:36.568511166 +0000 UTC m=+86.010594618"
Jul 22 15:14:36.569756 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.569680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-28qwf" event={"ID":"b16d0e5c-2b37-48f1-9641-7612e14b2b12","Type":"ContainerStarted","Data":"d3564f8867fd3f012ef8c32538822bc33cde6d396b40aa20a764a8407722c484"}
Jul 22 15:14:36.570185 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.570161    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:14:36.593516 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.592793    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702" exitCode=0
Jul 22 15:14:36.593758 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.592857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"}
Jul 22 15:14:36.616403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.616330    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-58d6n" podStartSLOduration=67.498630709 podStartE2EDuration="1m25.616306993s" totalImagesPullingTime="18.117676284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.569204455 +0000 UTC m=+86.011287903" watchObservedRunningTime="2026-07-22 15:14:36.616306993 +0000 UTC m=+86.058390440"
Jul 22 15:14:36.666913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.666802    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4lwsx" podStartSLOduration=67.619385556 podStartE2EDuration="1m25.66677197s" totalImagesPullingTime="18.047386414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.634443977 +0000 UTC m=+86.076527426" watchObservedRunningTime="2026-07-22 15:14:36.66677197 +0000 UTC m=+86.108855395"
Jul 22 15:14:36.668227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.667231    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-56v56" podStartSLOduration=3.34431557 podStartE2EDuration="21.66722299s" totalImagesPullingTime="18.32290742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.658613432 +0000 UTC m=+86.100696880" watchObservedRunningTime="2026-07-22 15:14:36.66722299 +0000 UTC m=+86.109306431"
Jul 22 15:14:36.702885 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.702700    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-28qwf" podStartSLOduration=67.527756525 podStartE2EDuration="1m25.702618806s" totalImagesPullingTime="18.174862281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.681019201 +0000 UTC m=+86.123102650" watchObservedRunningTime="2026-07-22 15:14:36.702618806 +0000 UTC m=+86.144702307"
Jul 22 15:14:36.705016 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.704931    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-6s5m6" podStartSLOduration=2.328747048 podStartE2EDuration="33.704912969s" totalImagesPullingTime="31.376165921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.702072761 +0000 UTC m=+86.144156210" watchObservedRunningTime="2026-07-22 15:14:36.704912969 +0000 UTC m=+86.146996417"
Jul 22 15:14:36.727619 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:36.726245    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2x9qs" podStartSLOduration=19.942087502 podStartE2EDuration="21.726218864s" totalImagesPullingTime="1.625786181s" totalInitContainerRuntime="158.345181ms" isStatefulPod=true podCreationTimestamp="2026-07-22 15:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:36.723984502 +0000 UTC m=+86.166067950" watchObservedRunningTime="2026-07-22 15:14:36.726218864 +0000 UTC m=+86.168302311"
Jul 22 15:14:46.687980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.687939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxqx6" event={"ID":"c8093da5-8487-48e9-94bf-ab78aefa884b","Type":"ContainerStarted","Data":"eb7c1be896b454c31ddaf6dffbb8a370d0cc3391b04e0e85ec115d4979b8f7aa"}
Jul 22 15:14:46.689669 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.689641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9" event={"ID":"ae7a5864-df9b-4d91-b451-5644b081226c","Type":"ContainerStarted","Data":"e7b4366e515a9c7f8f4c2b0fb79d9ad3600bde67420b3f588c8308c9065e6e0b"}
Jul 22 15:14:46.693779 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.693745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"}
Jul 22 15:14:46.693914 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.693831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"}
Jul 22 15:14:46.693914 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.693874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"}
Jul 22 15:14:46.693914 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.693912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"}
Jul 22 15:14:46.697713 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.697681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"1ae724c0b40543578ade37b871429b973dfb3686eb80a368024f499ab3596942"}
Jul 22 15:14:46.697907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.697891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"c197cd0d8a56190b256615ebcf390628f376288eadeaa0bc1352288f0933af23"}
Jul 22 15:14:46.698042 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.698030    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:46.698202 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.698165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" event={"ID":"a3e3074a-2f4f-47e5-99e9-a5646ea37d05","Type":"ContainerStarted","Data":"b10d3a2d1772d784a8269ad2547f177fc55f61bfd6c03baed9d7bfedfa38331b"}
Jul 22 15:14:46.699687 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.699637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz" event={"ID":"b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2","Type":"ContainerStarted","Data":"f8bdfb2f131498d58e92dc83bd79ab2737febabee38d5d6b80ca44bcecfecbee"}
Jul 22 15:14:46.700088 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.700068    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:46.704453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.704429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm"
Jul 22 15:14:46.706476 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.706452    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz"
Jul 22 15:14:46.707754 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.707731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"}
Jul 22 15:14:46.707939 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.707923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"}
Jul 22 15:14:46.708060 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.708046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"}
Jul 22 15:14:46.708218 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.708203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"}
Jul 22 15:14:46.708556 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.708427    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9" podStartSLOduration=17.705153126 podStartE2EDuration="27.708411729s" totalImagesPullingTime="10.003258603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:46.707291676 +0000 UTC m=+96.149375124" watchObservedRunningTime="2026-07-22 15:14:46.708411729 +0000 UTC m=+96.150495176"
Jul 22 15:14:46.731380 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.731342    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-69b988cc46-6fzsm" podStartSLOduration=3.02454681 podStartE2EDuration="29.731325512s" totalImagesPullingTime="26.706778702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:46.728653684 +0000 UTC m=+96.170737126" watchObservedRunningTime="2026-07-22 15:14:46.731325512 +0000 UTC m=+96.173408977"
Jul 22 15:14:46.751433 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:46.751384    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qdnvz" podStartSLOduration=16.479732296 podStartE2EDuration="26.751365737s" totalImagesPullingTime="10.271633441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:46.749317799 +0000 UTC m=+96.191401238" watchObservedRunningTime="2026-07-22 15:14:46.751365737 +0000 UTC m=+96.193449183"
Jul 22 15:14:47.720367 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:47.720328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"}
Jul 22 15:14:47.721026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:47.720416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerStarted","Data":"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"}
Jul 22 15:14:47.729540 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:47.728541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"}
Jul 22 15:14:47.729540 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:47.728613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerStarted","Data":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"}
Jul 22 15:14:47.754617 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:47.754556    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.58319113 podStartE2EDuration="31.75453343s" totalImagesPullingTime="27.153672591s" totalInitContainerRuntime="17.669709ms" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:47.748184636 +0000 UTC m=+97.190268102" watchObservedRunningTime="2026-07-22 15:14:47.75453343 +0000 UTC m=+97.196616880"
Jul 22 15:14:47.785027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:47.784404    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=17.503155876 podStartE2EDuration="26.784382369s" totalImagesPullingTime="9.211881385s" totalInitContainerRuntime="69.345108ms" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:47.778197708 +0000 UTC m=+97.220281180" watchObservedRunningTime="2026-07-22 15:14:47.784382369 +0000 UTC m=+97.226465817"
Jul 22 15:14:49.477509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:49.477449    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d566bd748-r22x7" podUID="b00fc07c-1bd4-4855-be6f-8800aa41cb16" containerName="console" containerID="cri-o://505883ac247892aedf3a60d17d400716bec79e5084b9f912cec8ed6db900c43b" gracePeriod=15
Jul 22 15:14:50.744013 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.743981    2580 generic.go:356] "Generic (PLEG): container finished" podID="b00fc07c-1bd4-4855-be6f-8800aa41cb16" containerID="505883ac247892aedf3a60d17d400716bec79e5084b9f912cec8ed6db900c43b" exitCode=0
Jul 22 15:14:50.744393 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.744033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d566bd748-r22x7" event={"ID":"b00fc07c-1bd4-4855-be6f-8800aa41cb16","Type":"ContainerDied","Data":"505883ac247892aedf3a60d17d400716bec79e5084b9f912cec8ed6db900c43b"}
Jul 22 15:14:50.808869 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.808833    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:50.917199 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917094    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcs6v\" (UniqueName: \"kubernetes.io/projected/b00fc07c-1bd4-4855-be6f-8800aa41cb16-kube-api-access-pcs6v\") pod \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") "
Jul 22 15:14:50.917199 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917177    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-config\") pod \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") "
Jul 22 15:14:50.917432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-service-ca\") pod \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") "
Jul 22 15:14:50.917432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917248    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-serving-cert\") pod \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") "
Jul 22 15:14:50.917432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-oauth-config\") pod \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") "
Jul 22 15:14:50.917432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-oauth-serving-cert\") pod \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\" (UID: \"b00fc07c-1bd4-4855-be6f-8800aa41cb16\") "
Jul 22 15:14:50.917732 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-service-ca" (OuterVolumeSpecName: "service-ca") pod "b00fc07c-1bd4-4855-be6f-8800aa41cb16" (UID: "b00fc07c-1bd4-4855-be6f-8800aa41cb16"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:50.917732 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917712    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-config" (OuterVolumeSpecName: "console-config") pod "b00fc07c-1bd4-4855-be6f-8800aa41cb16" (UID: "b00fc07c-1bd4-4855-be6f-8800aa41cb16"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:50.918070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.917841    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b00fc07c-1bd4-4855-be6f-8800aa41cb16" (UID: "b00fc07c-1bd4-4855-be6f-8800aa41cb16"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:50.919896 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.919869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00fc07c-1bd4-4855-be6f-8800aa41cb16-kube-api-access-pcs6v" (OuterVolumeSpecName: "kube-api-access-pcs6v") pod "b00fc07c-1bd4-4855-be6f-8800aa41cb16" (UID: "b00fc07c-1bd4-4855-be6f-8800aa41cb16"). InnerVolumeSpecName "kube-api-access-pcs6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:14:50.919896 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.919875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b00fc07c-1bd4-4855-be6f-8800aa41cb16" (UID: "b00fc07c-1bd4-4855-be6f-8800aa41cb16"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:14:50.920042 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:50.919960    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b00fc07c-1bd4-4855-be6f-8800aa41cb16" (UID: "b00fc07c-1bd4-4855-be6f-8800aa41cb16"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:14:51.018063 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.018030    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:51.018063 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.018059    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:51.018063 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.018068    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-serving-cert\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:51.018359 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.018078    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fc07c-1bd4-4855-be6f-8800aa41cb16-console-oauth-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:51.018359 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.018088    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fc07c-1bd4-4855-be6f-8800aa41cb16-oauth-serving-cert\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:51.018359 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.018096    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcs6v\" (UniqueName: \"kubernetes.io/projected/b00fc07c-1bd4-4855-be6f-8800aa41cb16-kube-api-access-pcs6v\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:51.749988 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.749957    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d566bd748-r22x7"
Jul 22 15:14:51.749988 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.749972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d566bd748-r22x7" event={"ID":"b00fc07c-1bd4-4855-be6f-8800aa41cb16","Type":"ContainerDied","Data":"04c8dcdca6a73a0611478a0bdb83c477d97d50f3d240e77417e178361f5ed18f"}
Jul 22 15:14:51.750527 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.750030    2580 scope.go:117] "RemoveContainer" containerID="505883ac247892aedf3a60d17d400716bec79e5084b9f912cec8ed6db900c43b"
Jul 22 15:14:51.752509 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.752457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxqx6" event={"ID":"c8093da5-8487-48e9-94bf-ab78aefa884b","Type":"ContainerStarted","Data":"63d5155b5861d2278d951f77a8744dbdc9d1f475788347f3efb1e857b36673e4"}
Jul 22 15:14:51.774531 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.774448    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lxqx6" podStartSLOduration=10.055055296 podStartE2EDuration="45.774430442s" totalImagesPullingTime="35.719375146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:14:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:51.773154149 +0000 UTC m=+101.215237599" watchObservedRunningTime="2026-07-22 15:14:51.774430442 +0000 UTC m=+101.216513920"
Jul 22 15:14:51.789528 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.789465    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d566bd748-r22x7"]
Jul 22 15:14:51.793198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:51.793169    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d566bd748-r22x7"]
Jul 22 15:14:52.286600 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:52.286566    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:14:53.021071 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:53.021034    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00fc07c-1bd4-4855-be6f-8800aa41cb16" path="/var/lib/kubelet/pods/b00fc07c-1bd4-4855-be6f-8800aa41cb16/volumes"
Jul 22 15:14:58.763720 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:58.763650    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7998484d86-fhxs6" podUID="8bf60db4-6230-4df0-a1f2-379a22fdc511" containerName="console" containerID="cri-o://9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b" gracePeriod=15
Jul 22 15:14:59.068204 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.068164    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:59.190123 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190089    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-service-ca\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190123 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190131    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-trusted-ca-bundle\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190149    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-oauth-serving-cert\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-config\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-serving-cert\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-oauth-config\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cft82\" (UniqueName: \"kubernetes.io/projected/8bf60db4-6230-4df0-a1f2-379a22fdc511-kube-api-access-cft82\") pod \"8bf60db4-6230-4df0-a1f2-379a22fdc511\" (UID: \"8bf60db4-6230-4df0-a1f2-379a22fdc511\") "
Jul 22 15:14:59.190659 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:59.190785 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:59.190865 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190806    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-config" (OuterVolumeSpecName: "console-config") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:59.190919 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.190871    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:14:59.192829 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.192803    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf60db4-6230-4df0-a1f2-379a22fdc511-kube-api-access-cft82" (OuterVolumeSpecName: "kube-api-access-cft82") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "kube-api-access-cft82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:14:59.192829 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.192808    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:14:59.192965 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.192855    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bf60db4-6230-4df0-a1f2-379a22fdc511" (UID: "8bf60db4-6230-4df0-a1f2-379a22fdc511"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291254    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cft82\" (UniqueName: \"kubernetes.io/projected/8bf60db4-6230-4df0-a1f2-379a22fdc511-kube-api-access-cft82\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291291    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291300    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-trusted-ca-bundle\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291308    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-oauth-serving-cert\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291317    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291326    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-serving-cert\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.291353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.291333    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf60db4-6230-4df0-a1f2-379a22fdc511-console-oauth-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:14:59.793221 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.793178    2580 generic.go:356] "Generic (PLEG): container finished" podID="8bf60db4-6230-4df0-a1f2-379a22fdc511" containerID="9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b" exitCode=0
Jul 22 15:14:59.793645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.793233    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7998484d86-fhxs6"
Jul 22 15:14:59.793645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.793266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7998484d86-fhxs6" event={"ID":"8bf60db4-6230-4df0-a1f2-379a22fdc511","Type":"ContainerDied","Data":"9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b"}
Jul 22 15:14:59.793645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.793334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7998484d86-fhxs6" event={"ID":"8bf60db4-6230-4df0-a1f2-379a22fdc511","Type":"ContainerDied","Data":"afc4d23e56fa3390c088200128d876d2e7365849a6481389f53ee4378cd17ff8"}
Jul 22 15:14:59.793645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.793354    2580 scope.go:117] "RemoveContainer" containerID="9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b"
Jul 22 15:14:59.808161 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.808138    2580 scope.go:117] "RemoveContainer" containerID="9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b"
Jul 22 15:14:59.808539 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:14:59.808516    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b\": container with ID starting with 9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b not found: ID does not exist" containerID="9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b"
Jul 22 15:14:59.808593 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.808550    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b"} err="failed to get container status \"9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b\": rpc error: code = NotFound desc = could not find container \"9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b\": container with ID starting with 9a603e86f2fbfe7333a24e72a4410a7fd9b87c7d00888332462b32a01f5a411b not found: ID does not exist"
Jul 22 15:14:59.813146 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.812960    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7998484d86-fhxs6"]
Jul 22 15:14:59.814958 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:14:59.814936    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7998484d86-fhxs6"]
Jul 22 15:15:00.340010 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:00.339973    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:15:00.340227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:00.340061    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:15:01.020147 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:01.020112    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf60db4-6230-4df0-a1f2-379a22fdc511" path="/var/lib/kubelet/pods/8bf60db4-6230-4df0-a1f2-379a22fdc511/volumes"
Jul 22 15:15:07.633878 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:07.633846    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-28qwf"
Jul 22 15:15:20.345996 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:20.345956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:15:20.349913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:20.349888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f789fd5c9-zfch9"
Jul 22 15:15:22.286081 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:22.286042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:22.341872 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:22.341838    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:22.913651 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:22.913623    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:36.029464 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.029430    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:15:36.030524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.030361    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-metric" containerID="cri-o://49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068" gracePeriod=120
Jul 22 15:15:36.030524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.030389    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="prom-label-proxy" containerID="cri-o://e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3" gracePeriod=120
Jul 22 15:15:36.030524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.030413    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-web" containerID="cri-o://a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874" gracePeriod=120
Jul 22 15:15:36.030524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.030435    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="config-reloader" containerID="cri-o://ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212" gracePeriod=120
Jul 22 15:15:36.030524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.030456    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy" containerID="cri-o://94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261" gracePeriod=120
Jul 22 15:15:36.030524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.030361    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="alertmanager" containerID="cri-o://b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc" gracePeriod=120
Jul 22 15:15:36.959907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.959875    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3" exitCode=0
Jul 22 15:15:36.959907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.959898    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261" exitCode=0
Jul 22 15:15:36.959907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.959906    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212" exitCode=0
Jul 22 15:15:36.959907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.959911    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc" exitCode=0
Jul 22 15:15:36.960169 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.959944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"}
Jul 22 15:15:36.960169 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.960006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"}
Jul 22 15:15:36.960169 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.960033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"}
Jul 22 15:15:36.960169 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:36.960057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"}
Jul 22 15:15:37.288572 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.288543    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:37.325769 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325698    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-main-db\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.325769 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325745    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-trusted-ca-bundle\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325787    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-main-tls\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325832    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-cluster-tls-config\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325861    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-web-config\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325889    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-web\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325932    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-metrics-client-ca\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.325974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-config-volume\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326028    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-metric\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326057    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkm9p\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-kube-api-access-hkm9p\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326076    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:15:37.326315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326086    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-tls-assets\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326161    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-config-out\") pod \"77d215ec-ddda-48d6-b726-f9165f3c5299\" (UID: \"77d215ec-ddda-48d6-b726-f9165f3c5299\") "
Jul 22 15:15:37.326739 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326413    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-main-db\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.326739 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326600    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:37.326837 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.326785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:37.331936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.331863    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-config-out" (OuterVolumeSpecName: "config-out") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:15:37.332333 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.332287    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.332800 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.332562    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.332800 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.332673    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:15:37.332800 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.332688    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.332977 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.332870    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-config-volume" (OuterVolumeSpecName: "config-volume") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.334208 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.334136    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.334329 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.334266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-kube-api-access-hkm9p" (OuterVolumeSpecName: "kube-api-access-hkm9p") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "kube-api-access-hkm9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:15:37.338364 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.338292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.346599 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.346547    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-web-config" (OuterVolumeSpecName: "web-config") pod "77d215ec-ddda-48d6-b726-f9165f3c5299" (UID: "77d215ec-ddda-48d6-b726-f9165f3c5299"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427482    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427685    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-main-tls\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427709    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-cluster-tls-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427725    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-web-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427749    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427763    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d215ec-ddda-48d6-b726-f9165f3c5299-metrics-client-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427777    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-config-volume\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427791    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427809    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkm9p\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-kube-api-access-hkm9p\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427823    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d215ec-ddda-48d6-b726-f9165f3c5299-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427836    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d215ec-ddda-48d6-b726-f9165f3c5299-tls-assets\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.431936 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.427849    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d215ec-ddda-48d6-b726-f9165f3c5299-config-out\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:37.967579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967542    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068" exitCode=0
Jul 22 15:15:37.967579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967574    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerID="a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874" exitCode=0
Jul 22 15:15:37.967813 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"}
Jul 22 15:15:37.967813 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967646    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:37.967813 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"}
Jul 22 15:15:37.967813 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77d215ec-ddda-48d6-b726-f9165f3c5299","Type":"ContainerDied","Data":"5a5353b3d7c0bfd3e818e4404ed9143f1ae02ed0be82a595648c7ce500e3b05c"}
Jul 22 15:15:37.967813 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.967795    2580 scope.go:117] "RemoveContainer" containerID="e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"
Jul 22 15:15:37.976262 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.976236    2580 scope.go:117] "RemoveContainer" containerID="49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"
Jul 22 15:15:37.984638 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.984616    2580 scope.go:117] "RemoveContainer" containerID="94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"
Jul 22 15:15:37.991878 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.991850    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:15:37.993046 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.993026    2580 scope.go:117] "RemoveContainer" containerID="a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"
Jul 22 15:15:37.996348 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:37.996320    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:15:38.001560 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.001537    2580 scope.go:117] "RemoveContainer" containerID="ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"
Jul 22 15:15:38.009342 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.009323    2580 scope.go:117] "RemoveContainer" containerID="b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"
Jul 22 15:15:38.017696 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.017676    2580 scope.go:117] "RemoveContainer" containerID="f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"
Jul 22 15:15:38.023763 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.023725    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:15:38.024212 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024187    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="prom-label-proxy"
Jul 22 15:15:38.024212 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024214    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="prom-label-proxy"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024228    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-metric"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024239    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-metric"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024253    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="alertmanager"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024262    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="alertmanager"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024286    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="config-reloader"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024294    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="config-reloader"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024308    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf60db4-6230-4df0-a1f2-379a22fdc511" containerName="console"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024317    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf60db4-6230-4df0-a1f2-379a22fdc511" containerName="console"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024328    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024336    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy"
Jul 22 15:15:38.024352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024349    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-web"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024360    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-web"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024376    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="init-config-reloader"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024385    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="init-config-reloader"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024399    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00fc07c-1bd4-4855-be6f-8800aa41cb16" containerName="console"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024406    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00fc07c-1bd4-4855-be6f-8800aa41cb16" containerName="console"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024504    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-web"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024518    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf60db4-6230-4df0-a1f2-379a22fdc511" containerName="console"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024528    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="alertmanager"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024539    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="config-reloader"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024548    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="prom-label-proxy"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024557    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024566    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" containerName="kube-rbac-proxy-metric"
Jul 22 15:15:38.024741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.024576    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00fc07c-1bd4-4855-be6f-8800aa41cb16" containerName="console"
Jul 22 15:15:38.028000 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.027977    2580 scope.go:117] "RemoveContainer" containerID="e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"
Jul 22 15:15:38.028314 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.028290    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3\": container with ID starting with e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3 not found: ID does not exist" containerID="e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"
Jul 22 15:15:38.028369 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.028325    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"} err="failed to get container status \"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3\": rpc error: code = NotFound desc = could not find container \"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3\": container with ID starting with e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3 not found: ID does not exist"
Jul 22 15:15:38.028369 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.028345    2580 scope.go:117] "RemoveContainer" containerID="49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"
Jul 22 15:15:38.028812 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.028786    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068\": container with ID starting with 49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068 not found: ID does not exist" containerID="49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"
Jul 22 15:15:38.028919 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.028821    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"} err="failed to get container status \"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068\": rpc error: code = NotFound desc = could not find container \"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068\": container with ID starting with 49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068 not found: ID does not exist"
Jul 22 15:15:38.028919 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.028846    2580 scope.go:117] "RemoveContainer" containerID="94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"
Jul 22 15:15:38.029058 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.029222 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.029147    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261\": container with ID starting with 94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261 not found: ID does not exist" containerID="94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"
Jul 22 15:15:38.029222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029176    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"} err="failed to get container status \"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261\": rpc error: code = NotFound desc = could not find container \"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261\": container with ID starting with 94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261 not found: ID does not exist"
Jul 22 15:15:38.029222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029190    2580 scope.go:117] "RemoveContainer" containerID="a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"
Jul 22 15:15:38.029526 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.029470    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874\": container with ID starting with a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874 not found: ID does not exist" containerID="a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"
Jul 22 15:15:38.029647 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029528    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"} err="failed to get container status \"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874\": rpc error: code = NotFound desc = could not find container \"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874\": container with ID starting with a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874 not found: ID does not exist"
Jul 22 15:15:38.029647 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029551    2580 scope.go:117] "RemoveContainer" containerID="ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"
Jul 22 15:15:38.029828 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.029807    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212\": container with ID starting with ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212 not found: ID does not exist" containerID="ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"
Jul 22 15:15:38.029896 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029836    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"} err="failed to get container status \"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212\": rpc error: code = NotFound desc = could not find container \"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212\": container with ID starting with ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212 not found: ID does not exist"
Jul 22 15:15:38.029896 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.029857    2580 scope.go:117] "RemoveContainer" containerID="b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"
Jul 22 15:15:38.030130 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.030114    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc\": container with ID starting with b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc not found: ID does not exist" containerID="b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"
Jul 22 15:15:38.030191 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030138    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"} err="failed to get container status \"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc\": rpc error: code = NotFound desc = could not find container \"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc\": container with ID starting with b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc not found: ID does not exist"
Jul 22 15:15:38.030191 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030156    2580 scope.go:117] "RemoveContainer" containerID="f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"
Jul 22 15:15:38.030403 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:38.030387    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702\": container with ID starting with f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702 not found: ID does not exist" containerID="f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"
Jul 22 15:15:38.030455 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030408    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"} err="failed to get container status \"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702\": rpc error: code = NotFound desc = could not find container \"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702\": container with ID starting with f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702 not found: ID does not exist"
Jul 22 15:15:38.030455 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030422    2580 scope.go:117] "RemoveContainer" containerID="e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"
Jul 22 15:15:38.030651 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030630    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3"} err="failed to get container status \"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3\": rpc error: code = NotFound desc = could not find container \"e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3\": container with ID starting with e80afe15b58a4821bd0475de62e8871c13ea5a4e7cae8a0d2bb38fc2ef0739c3 not found: ID does not exist"
Jul 22 15:15:38.030697 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030653    2580 scope.go:117] "RemoveContainer" containerID="49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"
Jul 22 15:15:38.030872 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030850    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068"} err="failed to get container status \"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068\": rpc error: code = NotFound desc = could not find container \"49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068\": container with ID starting with 49f30074cae9ad2ba372182e31f9b701d34f884b58cb9140cecffac5aa239068 not found: ID does not exist"
Jul 22 15:15:38.030950 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.030873    2580 scope.go:117] "RemoveContainer" containerID="94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"
Jul 22 15:15:38.031096 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031073    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261"} err="failed to get container status \"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261\": rpc error: code = NotFound desc = could not find container \"94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261\": container with ID starting with 94ecf58b305cd876017741c32872a2532591d946bd9ec2b40b714c8f1a930261 not found: ID does not exist"
Jul 22 15:15:38.031149 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031097    2580 scope.go:117] "RemoveContainer" containerID="a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"
Jul 22 15:15:38.031309 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031287    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874"} err="failed to get container status \"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874\": rpc error: code = NotFound desc = could not find container \"a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874\": container with ID starting with a1e0222263b96d3b6ab2c2f37d13beac45e93c0ab036f39026f58c4e2dc6f874 not found: ID does not exist"
Jul 22 15:15:38.031366 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031309    2580 scope.go:117] "RemoveContainer" containerID="ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031717    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212"} err="failed to get container status \"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212\": rpc error: code = NotFound desc = could not find container \"ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212\": container with ID starting with ecac5d27aaf11a529a7ba723f8d1ca116b2652759b065f63f556523f53e57212 not found: ID does not exist"
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031741    2580 scope.go:117] "RemoveContainer" containerID="b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031731    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031817    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rjq9p\""
Jul 22 15:15:38.031913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.031812    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 15:15:38.032368 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.032092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 15:15:38.032368 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.032175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 15:15:38.032368 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.032195    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 15:15:38.032368 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.032314    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc"} err="failed to get container status \"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc\": rpc error: code = NotFound desc = could not find container \"b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc\": container with ID starting with b056f44824a2406c6d0c6f2dde8c69713dd6942c9a87734ae245d11d002189dc not found: ID does not exist"
Jul 22 15:15:38.032368 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.032339    2580 scope.go:117] "RemoveContainer" containerID="f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"
Jul 22 15:15:38.032691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.032672    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702"} err="failed to get container status \"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702\": rpc error: code = NotFound desc = could not find container \"f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702\": container with ID starting with f2c8144143abfebf1d194665fcea19d66879c5d0191c643d9af9d051ffdfa702 not found: ID does not exist"
Jul 22 15:15:38.037064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.037042    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 15:15:38.045391 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.045360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:15:38.132139 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04afe23-cdfe-48dd-affe-8b4ed48039e9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132139 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-config-volume\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04afe23-cdfe-48dd-affe-8b4ed48039e9-config-out\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997pj\" (UniqueName: \"kubernetes.io/projected/c04afe23-cdfe-48dd-affe-8b4ed48039e9-kube-api-access-997pj\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04afe23-cdfe-48dd-affe-8b4ed48039e9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-web-config\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04afe23-cdfe-48dd-affe-8b4ed48039e9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.132555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.132550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04afe23-cdfe-48dd-affe-8b4ed48039e9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233499 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233499 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233499 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04afe23-cdfe-48dd-affe-8b4ed48039e9-config-out\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233735 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997pj\" (UniqueName: \"kubernetes.io/projected/c04afe23-cdfe-48dd-affe-8b4ed48039e9-kube-api-access-997pj\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233735 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04afe23-cdfe-48dd-affe-8b4ed48039e9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233735 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-web-config\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04afe23-cdfe-48dd-affe-8b4ed48039e9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.233876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04afe23-cdfe-48dd-affe-8b4ed48039e9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.234104 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04afe23-cdfe-48dd-affe-8b4ed48039e9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.234104 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-config-volume\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.234104 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.233960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.235222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.234512    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c04afe23-cdfe-48dd-affe-8b4ed48039e9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.235222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.234945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c04afe23-cdfe-48dd-affe-8b4ed48039e9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.235222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.234952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c04afe23-cdfe-48dd-affe-8b4ed48039e9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.236886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.236811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.236886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.236823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c04afe23-cdfe-48dd-affe-8b4ed48039e9-config-out\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.237277 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.237255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.237632 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.237610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.238028 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.238009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.238109 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.238013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.238336 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.238315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-config-volume\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.238472 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.238455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c04afe23-cdfe-48dd-affe-8b4ed48039e9-web-config\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.238627 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.238613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c04afe23-cdfe-48dd-affe-8b4ed48039e9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.242230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.242207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-997pj\" (UniqueName: \"kubernetes.io/projected/c04afe23-cdfe-48dd-affe-8b4ed48039e9-kube-api-access-997pj\") pod \"alertmanager-main-0\" (UID: \"c04afe23-cdfe-48dd-affe-8b4ed48039e9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.339039 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.338996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:15:38.487006 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.486914    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:15:38.489988 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:15:38.489956    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04afe23_cdfe_48dd_affe_8b4ed48039e9.slice/crio-85f3b625d8f516d34f2affe690a3d6d3fbdefb08b780fefcc51603587dfce9e3 WatchSource:0}: Error finding container 85f3b625d8f516d34f2affe690a3d6d3fbdefb08b780fefcc51603587dfce9e3: Status 404 returned error can't find the container with id 85f3b625d8f516d34f2affe690a3d6d3fbdefb08b780fefcc51603587dfce9e3
Jul 22 15:15:38.973263 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.973228    2580 generic.go:356] "Generic (PLEG): container finished" podID="c04afe23-cdfe-48dd-affe-8b4ed48039e9" containerID="83510dddeb11a425677d7f690213f407d1c0950a5f3eef7a039283812f39acf5" exitCode=0
Jul 22 15:15:38.973432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.973315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerDied","Data":"83510dddeb11a425677d7f690213f407d1c0950a5f3eef7a039283812f39acf5"}
Jul 22 15:15:38.973432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:38.973395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"85f3b625d8f516d34f2affe690a3d6d3fbdefb08b780fefcc51603587dfce9e3"}
Jul 22 15:15:39.021731 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.021694    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77d215ec-ddda-48d6-b726-f9165f3c5299" path="/var/lib/kubelet/pods/77d215ec-ddda-48d6-b726-f9165f3c5299/volumes"
Jul 22 15:15:39.982138 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.982096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"2203e7590b5d92e83d6c2bb84f8ef88594850380dcfe835fd1df59891389fd51"}
Jul 22 15:15:39.982650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.982205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"3fa6a1c4dd6250abc49a1a49f203983ca172c07d39e1db52fdf2053de66c5c0a"}
Jul 22 15:15:39.982650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.982250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"954dc7a08b65952b278b7270f2ca518174a2720372a36034759e971941537a7f"}
Jul 22 15:15:39.982650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.982289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"6740b7f59dde64f2f7f210617fb081e7d127dbb7eb935f34415ba5316099b357"}
Jul 22 15:15:39.982650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.982324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"3db062c4e940b88cc4f82084176d61b9861ff5eb27eb4c59d55982152a6840bb"}
Jul 22 15:15:39.982650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:39.982360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c04afe23-cdfe-48dd-affe-8b4ed48039e9","Type":"ContainerStarted","Data":"e3545916ee4a4405f1c346d55f9000a2ecd55d42bc3b52674539ced44fe5e6c3"}
Jul 22 15:15:40.011972 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.011902    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.9970184739999999 podStartE2EDuration="2.011879709s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.861235ms" isStatefulPod=false podCreationTimestamp="2026-07-22 15:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:15:40.010433511 +0000 UTC m=+149.452516985" watchObservedRunningTime="2026-07-22 15:15:40.011879709 +0000 UTC m=+149.453963158"
Jul 22 15:15:40.376839 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.376792    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:15:40.377426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.377183    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="prometheus" containerID="cri-o://29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761" gracePeriod=600
Jul 22 15:15:40.377426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.377194    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy" containerID="cri-o://229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23" gracePeriod=600
Jul 22 15:15:40.377426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.377233    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-web" containerID="cri-o://8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160" gracePeriod=600
Jul 22 15:15:40.377426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.377259    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678" gracePeriod=600
Jul 22 15:15:40.377426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.377206    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="thanos-sidecar" containerID="cri-o://f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676" gracePeriod=600
Jul 22 15:15:40.377426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.377353    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="config-reloader" containerID="cri-o://7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9" gracePeriod=600
Jul 22 15:15:40.641443 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.641412    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:40.759416 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759382    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-2\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759416 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-1\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-db\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759559    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-kubelet-serving-ca-bundle\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759575    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-metrics-client-ca\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759601    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-thanos-prometheus-http-client-file\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-serving-certs-ca-bundle\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs5l7\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-kube-api-access-bs5l7\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.759699 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-tls-assets\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759710    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-metrics-client-certs\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-config-out\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759783    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-config\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-grpc-tls\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-0\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-tls\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759948    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-trusted-ca-bundle\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.759975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-web-config\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760004    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-kube-rbac-proxy\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760033    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f17b0122-1170-42dc-88b4-1e111e0af6b4\" (UID: \"f17b0122-1170-42dc-88b4-1e111e0af6b4\") "
Jul 22 15:15:40.760070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760043    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.760712 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.760712 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760581    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.760712 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.760638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:15:40.761764 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761738    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.761944 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761767    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.761944 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761782    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-db\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.761944 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761795    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.761944 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761809    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-metrics-client-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.761944 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761822    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.761944 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.761835    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.762396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.762354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:15:40.764829 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.764758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.764829 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.764789    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:15:40.765033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.764900    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-kube-api-access-bs5l7" (OuterVolumeSpecName: "kube-api-access-bs5l7") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "kube-api-access-bs5l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:15:40.765033 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.764911    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.765158 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765134    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.765257 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765193    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-config-out" (OuterVolumeSpecName: "config-out") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:15:40.765311 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765288    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.765359 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.765416 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765375    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-config" (OuterVolumeSpecName: "config") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.765511 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765458    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.765563 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.765513    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.775880 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.775847    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-web-config" (OuterVolumeSpecName: "web-config") pod "f17b0122-1170-42dc-88b4-1e111e0af6b4" (UID: "f17b0122-1170-42dc-88b4-1e111e0af6b4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:40.863100 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863050    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863100 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863088    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863100 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863098    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs5l7\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-kube-api-access-bs5l7\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863100 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863109    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17b0122-1170-42dc-88b4-1e111e0af6b4-tls-assets\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863100 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863117    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-metrics-client-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863125    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17b0122-1170-42dc-88b4-1e111e0af6b4-config-out\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863133    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863140    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-grpc-tls\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863148    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17b0122-1170-42dc-88b4-1e111e0af6b4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863155    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863164    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-web-config\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863173    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-kube-rbac-proxy\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.863395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.863183    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17b0122-1170-42dc-88b4-1e111e0af6b4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:15:40.990112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990016    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678" exitCode=0
Jul 22 15:15:40.990112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990043    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23" exitCode=0
Jul 22 15:15:40.990112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990049    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160" exitCode=0
Jul 22 15:15:40.990112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990055    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676" exitCode=0
Jul 22 15:15:40.990112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990060    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9" exitCode=0
Jul 22 15:15:40.990112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990065    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761" exitCode=0
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"}
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990117    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990203    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"}
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"}
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"}
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"}
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"}
Jul 22 15:15:40.990705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:40.990535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17b0122-1170-42dc-88b4-1e111e0af6b4","Type":"ContainerDied","Data":"66096668fae8b048b8490fd021800864056e9285ceeaabcc8e4171f892f5e4b4"}
Jul 22 15:15:41.000048 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.000027    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.009178 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.009148    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.018772 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.018384    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.021620 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.021598    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:15:41.021709 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.021636    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:15:41.026712 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.026689    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.039069 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.039039    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.045950 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.045910    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:15:41.046249 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046234    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="config-reloader"
Jul 22 15:15:41.046249 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046248    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="config-reloader"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046260    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046266    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046273    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-thanos"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046279    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-thanos"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046286    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="thanos-sidecar"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046291    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="thanos-sidecar"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046306    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="init-config-reloader"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046311    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="init-config-reloader"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046317    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="prometheus"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046322    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="prometheus"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046328    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-web"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046333    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-web"
Jul 22 15:15:41.046385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046391    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="config-reloader"
Jul 22 15:15:41.046986 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046399    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-thanos"
Jul 22 15:15:41.046986 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046406    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="prometheus"
Jul 22 15:15:41.046986 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046412    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="thanos-sidecar"
Jul 22 15:15:41.046986 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046417    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy"
Jul 22 15:15:41.046986 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.046423    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" containerName="kube-rbac-proxy-web"
Jul 22 15:15:41.048910 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.048886    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.050396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.050377    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.054354 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.053782    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 15:15:41.054354 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.054037    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 15:15:41.054354 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.054235    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 15:15:41.054690 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.054431    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 15:15:41.055114 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.054914    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 15:15:41.055114 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.054930    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 15:15:41.055114 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.054949    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5kfpuudmhq6r\""
Jul 22 15:15:41.055298 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.055196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 15:15:41.055355 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.055296    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 15:15:41.055771 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.055569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 15:15:41.055771 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.055577    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 15:15:41.055771 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.055692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kxk4d\""
Jul 22 15:15:41.055981 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.055845    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 15:15:41.056202 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.056178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 15:15:41.057433 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.057412    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 15:15:41.059748 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.059726    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 15:15:41.060792 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.060774    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.061128 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.061105    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.061175 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.061143    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"} err="failed to get container status \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist"
Jul 22 15:15:41.061175 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.061169    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.061434 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.061415    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.061565 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.061437    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"} err="failed to get container status \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist"
Jul 22 15:15:41.061565 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.061452    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.061771 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.061755    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.061818 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.061776    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"} err="failed to get container status \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist"
Jul 22 15:15:41.061818 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.061792    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.062028 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.062010    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.062071 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.062034    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"} err="failed to get container status \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist"
Jul 22 15:15:41.062071 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.062050    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.062307 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.062291    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.062346 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.062311    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"} err="failed to get container status \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist"
Jul 22 15:15:41.062346 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.062323    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.062804 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.062784    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.062849 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.062808    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"} err="failed to get container status \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist"
Jul 22 15:15:41.062849 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.062822    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.063058 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:15:41.063040    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.063106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.063062    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"} err="failed to get container status \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist"
Jul 22 15:15:41.063106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.063076    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.063315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.063287    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"} err="failed to get container status \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist"
Jul 22 15:15:41.063315 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.063315    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.063771 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.063745    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"} err="failed to get container status \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist"
Jul 22 15:15:41.063856 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.063773    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.064090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.064068    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"} err="failed to get container status \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist"
Jul 22 15:15:41.064155 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.064093    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.064343 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.064314    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"} err="failed to get container status \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist"
Jul 22 15:15:41.064404 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.064347    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.065259 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.064623    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"} err="failed to get container status \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist"
Jul 22 15:15:41.065259 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.064655    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.065259 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.065161    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"} err="failed to get container status \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist"
Jul 22 15:15:41.065259 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.065186    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.065536 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.065516    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"} err="failed to get container status \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist"
Jul 22 15:15:41.065605 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.065542    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.066048 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066029    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"} err="failed to get container status \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist"
Jul 22 15:15:41.066048 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066048    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.066184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066144    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:15:41.066330 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066307    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"} err="failed to get container status \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist"
Jul 22 15:15:41.066377 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066335    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.066602 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066575    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"} err="failed to get container status \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist"
Jul 22 15:15:41.066691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066603    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.066877 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066859    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"} err="failed to get container status \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist"
Jul 22 15:15:41.066930 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.066878    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.067112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.067095    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"} err="failed to get container status \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist"
Jul 22 15:15:41.067152 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.067114    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.067416 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.067387    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"} err="failed to get container status \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist"
Jul 22 15:15:41.067541 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.067419    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.067869 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.067848    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"} err="failed to get container status \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist"
Jul 22 15:15:41.067869 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.067869    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.068142 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068123    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"} err="failed to get container status \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist"
Jul 22 15:15:41.068221 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068143    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.068445 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068423    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"} err="failed to get container status \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist"
Jul 22 15:15:41.068445 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068445    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.068713 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068690    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"} err="failed to get container status \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist"
Jul 22 15:15:41.068773 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068716    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.069013 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.068987    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"} err="failed to get container status \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist"
Jul 22 15:15:41.069013 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069012    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.069292 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069272    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"} err="failed to get container status \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist"
Jul 22 15:15:41.069292 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069292    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.069713 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069688    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"} err="failed to get container status \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist"
Jul 22 15:15:41.069800 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069713    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.069953 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069937    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"} err="failed to get container status \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist"
Jul 22 15:15:41.069953 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.069953    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.070161 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070141    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"} err="failed to get container status \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist"
Jul 22 15:15:41.070226 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070163    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.070358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070341    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"} err="failed to get container status \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist"
Jul 22 15:15:41.070421 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070357    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.070613 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070595    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"} err="failed to get container status \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist"
Jul 22 15:15:41.070613 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070611    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.070814 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070797    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"} err="failed to get container status \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist"
Jul 22 15:15:41.070886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.070812    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.071036 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071018    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"} err="failed to get container status \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist"
Jul 22 15:15:41.071036 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071036    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.071230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071207    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"} err="failed to get container status \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist"
Jul 22 15:15:41.071230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071230    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.071537 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071520    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"} err="failed to get container status \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist"
Jul 22 15:15:41.071537 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071537    2580 scope.go:117] "RemoveContainer" containerID="7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"
Jul 22 15:15:41.071737 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071716    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678"} err="failed to get container status \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": rpc error: code = NotFound desc = could not find container \"7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678\": container with ID starting with 7d6a2bc916637dd0ffc848de207a0de3f11d5e890c30fe5f32b0a410a7feb678 not found: ID does not exist"
Jul 22 15:15:41.071737 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071737    2580 scope.go:117] "RemoveContainer" containerID="229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"
Jul 22 15:15:41.071962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071929    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23"} err="failed to get container status \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": rpc error: code = NotFound desc = could not find container \"229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23\": container with ID starting with 229f51fc6dd8c04166ec40281a2ec63c01d6ab722e4249a8a29103f2f362df23 not found: ID does not exist"
Jul 22 15:15:41.071962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.071942    2580 scope.go:117] "RemoveContainer" containerID="8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"
Jul 22 15:15:41.072117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.072097    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160"} err="failed to get container status \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": rpc error: code = NotFound desc = could not find container \"8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160\": container with ID starting with 8723bbd969edac1513bd8b93586b27c0ee8784213660609adf1897cfbed6d160 not found: ID does not exist"
Jul 22 15:15:41.072117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.072116    2580 scope.go:117] "RemoveContainer" containerID="f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"
Jul 22 15:15:41.074985 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.074946    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676"} err="failed to get container status \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": rpc error: code = NotFound desc = could not find container \"f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676\": container with ID starting with f40fa463d9d36941070aeeff4621d19f5b73153c3e20967705927da8b6956676 not found: ID does not exist"
Jul 22 15:15:41.074985 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.074984    2580 scope.go:117] "RemoveContainer" containerID="7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"
Jul 22 15:15:41.075562 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.075534    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9"} err="failed to get container status \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": rpc error: code = NotFound desc = could not find container \"7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9\": container with ID starting with 7b14c712bdebc27d3f49b508019a7214993b3c061cb55a555453b49a389d08c9 not found: ID does not exist"
Jul 22 15:15:41.075562 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.075565    2580 scope.go:117] "RemoveContainer" containerID="29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"
Jul 22 15:15:41.075887 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.075864    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761"} err="failed to get container status \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": rpc error: code = NotFound desc = could not find container \"29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761\": container with ID starting with 29c745481f0edb1c0e0cb83b8979c298a8d7d45af898706c44df8970fcd6f761 not found: ID does not exist"
Jul 22 15:15:41.075982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.075889    2580 scope.go:117] "RemoveContainer" containerID="28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"
Jul 22 15:15:41.076169 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.076140    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8"} err="failed to get container status \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": rpc error: code = NotFound desc = could not find container \"28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8\": container with ID starting with 28c20d61167a9ecea5f2859ecad04c0c023616c109a73249e4804d24d04e78b8 not found: ID does not exist"
Jul 22 15:15:41.165396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-web-config\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9940605a-1343-4c4b-b65a-939da48e9032-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165664 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9940605a-1343-4c4b-b65a-939da48e9032-config-out\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdb42\" (UniqueName: \"kubernetes.io/projected/9940605a-1343-4c4b-b65a-939da48e9032-kube-api-access-qdb42\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-config\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165824    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.165918 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.166191 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.165906    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.266779 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.266742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.266779 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.266786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.266982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.266813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.266982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.266940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267044 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.266996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267202 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267255 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267302 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267302 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-web-config\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267394 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267394 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9940605a-1343-4c4b-b65a-939da48e9032-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267483 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267483 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9940605a-1343-4c4b-b65a-939da48e9032-config-out\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267483 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267483 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdb42\" (UniqueName: \"kubernetes.io/projected/9940605a-1343-4c4b-b65a-939da48e9032-kube-api-access-qdb42\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-config\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.267976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.267727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.268165 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.268011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.268298 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.268200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.268298 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.268279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.270759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.270623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9940605a-1343-4c4b-b65a-939da48e9032-config-out\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.270911 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.270825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.270911 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.270855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9940605a-1343-4c4b-b65a-939da48e9032-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.271066 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.271037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-config\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.271331 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.271308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.271422 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.271367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.271739 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.271547    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-web-config\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.272122 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.272098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.272221 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.272195    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940605a-1343-4c4b-b65a-939da48e9032-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.273205 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.273175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.273606 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.273586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.273685 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.273666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.273770 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.273751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9940605a-1343-4c4b-b65a-939da48e9032-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.280311 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.280282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdb42\" (UniqueName: \"kubernetes.io/projected/9940605a-1343-4c4b-b65a-939da48e9032-kube-api-access-qdb42\") pod \"prometheus-k8s-0\" (UID: \"9940605a-1343-4c4b-b65a-939da48e9032\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.363327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.363282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:15:41.511689 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.511657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:15:41.514594 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:15:41.514568    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9940605a_1343_4c4b_b65a_939da48e9032.slice/crio-8d91742118b0fa5bb0769a0887862f9caae1aa6b71b34f54ec23f432864b309b WatchSource:0}: Error finding container 8d91742118b0fa5bb0769a0887862f9caae1aa6b71b34f54ec23f432864b309b: Status 404 returned error can't find the container with id 8d91742118b0fa5bb0769a0887862f9caae1aa6b71b34f54ec23f432864b309b
Jul 22 15:15:41.997120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.997084    2580 generic.go:356] "Generic (PLEG): container finished" podID="9940605a-1343-4c4b-b65a-939da48e9032" containerID="4969b8802bd1ca9c096491d838cf100e24d222335e1e76e425e2c8ee196d4dc7" exitCode=0
Jul 22 15:15:41.997626 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.997141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerDied","Data":"4969b8802bd1ca9c096491d838cf100e24d222335e1e76e425e2c8ee196d4dc7"}
Jul 22 15:15:41.997626 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:41.997193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"8d91742118b0fa5bb0769a0887862f9caae1aa6b71b34f54ec23f432864b309b"}
Jul 22 15:15:43.004761 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.004723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"04a6145182bb5d152406bfee6c6a6313fd5271489199b1fbb1f609a028984f3c"}
Jul 22 15:15:43.005248 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.004827    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"59518e42ce6cc3c055e6d4b433151b134126942f9168ea1fd8f3aacc6d850cb2"}
Jul 22 15:15:43.005248 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.004853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"6a32a3c00deb8696abba7993916f5bba0bf96fa5374423fd75f186ff0f939495"}
Jul 22 15:15:43.005248 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.004878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"35b375ef037867d09b211c8cc268e81b8b1a598662f75e1632d43c73003757ed"}
Jul 22 15:15:43.005248 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.004900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"f6b6c861beb264e99b5843eeee5d5f6675d38757e5878033dbff21e0d5e86679"}
Jul 22 15:15:43.005248 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.004925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9940605a-1343-4c4b-b65a-939da48e9032","Type":"ContainerStarted","Data":"e736f7310021cde7ac4b4a0790cc2d6accda21841d36690d8cb1e24e0d4eef79"}
Jul 22 15:15:43.020430 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.020395    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f17b0122-1170-42dc-88b4-1e111e0af6b4" path="/var/lib/kubelet/pods/f17b0122-1170-42dc-88b4-1e111e0af6b4/volumes"
Jul 22 15:15:43.035540 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:43.035458    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.016870191 podStartE2EDuration="2.035434102s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.563911ms" isStatefulPod=false podCreationTimestamp="2026-07-22 15:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:15:43.033249462 +0000 UTC m=+152.475332908" watchObservedRunningTime="2026-07-22 15:15:43.035434102 +0000 UTC m=+152.477517570"
Jul 22 15:15:46.364351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:15:46.364312    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:16:41.364209 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:16:41.364166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:16:41.380760 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:16:41.380734    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:16:42.252746 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:16:42.252708    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:18:10.927093 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:10.927060    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 15:18:36.758505 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.758461    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-q76rq"]
Jul 22 15:18:36.761757 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.761741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:36.764469 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.764448    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 22 15:18:36.764626 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.764478    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 22 15:18:36.772225 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.772204    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-q76rq"]
Jul 22 15:18:36.837367 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.837318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b179c4a7-066a-4e57-bdf8-ddea347283ca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-q76rq\" (UID: \"b179c4a7-066a-4e57-bdf8-ddea347283ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:36.837592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.837415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t65bk\" (UniqueName: \"kubernetes.io/projected/b179c4a7-066a-4e57-bdf8-ddea347283ca-kube-api-access-t65bk\") pod \"cert-manager-webhook-6486ffd997-q76rq\" (UID: \"b179c4a7-066a-4e57-bdf8-ddea347283ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:36.938426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.938388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t65bk\" (UniqueName: \"kubernetes.io/projected/b179c4a7-066a-4e57-bdf8-ddea347283ca-kube-api-access-t65bk\") pod \"cert-manager-webhook-6486ffd997-q76rq\" (UID: \"b179c4a7-066a-4e57-bdf8-ddea347283ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:36.938426 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.938434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b179c4a7-066a-4e57-bdf8-ddea347283ca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-q76rq\" (UID: \"b179c4a7-066a-4e57-bdf8-ddea347283ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:36.950293 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.950264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t65bk\" (UniqueName: \"kubernetes.io/projected/b179c4a7-066a-4e57-bdf8-ddea347283ca-kube-api-access-t65bk\") pod \"cert-manager-webhook-6486ffd997-q76rq\" (UID: \"b179c4a7-066a-4e57-bdf8-ddea347283ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:36.953185 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:36.953157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b179c4a7-066a-4e57-bdf8-ddea347283ca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-q76rq\" (UID: \"b179c4a7-066a-4e57-bdf8-ddea347283ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:37.087054 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:37.086956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:37.227777 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:37.227739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-q76rq"]
Jul 22 15:18:37.230827 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:18:37.230800    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb179c4a7_066a_4e57_bdf8_ddea347283ca.slice/crio-cff3de0257dbdf6dc58bfb40c49b7ef3e3ea879922f80fbd344d605cae4d2258 WatchSource:0}: Error finding container cff3de0257dbdf6dc58bfb40c49b7ef3e3ea879922f80fbd344d605cae4d2258: Status 404 returned error can't find the container with id cff3de0257dbdf6dc58bfb40c49b7ef3e3ea879922f80fbd344d605cae4d2258
Jul 22 15:18:37.233411 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:37.233389    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:18:37.692729 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:37.692691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq" event={"ID":"b179c4a7-066a-4e57-bdf8-ddea347283ca","Type":"ContainerStarted","Data":"cff3de0257dbdf6dc58bfb40c49b7ef3e3ea879922f80fbd344d605cae4d2258"}
Jul 22 15:18:40.709671 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:40.709625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq" event={"ID":"b179c4a7-066a-4e57-bdf8-ddea347283ca","Type":"ContainerStarted","Data":"e89d2e10409145236c56caee63c02dd7609af7af788da867465d7e4347fbb77c"}
Jul 22 15:18:40.710306 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:40.709696    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:40.729182 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:40.729129    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq" podStartSLOduration=1.349930073 podStartE2EDuration="4.729113377s" totalImagesPullingTime="3.379183304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:18:40.727266403 +0000 UTC m=+330.169349851" watchObservedRunningTime="2026-07-22 15:18:40.729113377 +0000 UTC m=+330.171196823"
Jul 22 15:18:46.715940 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:46.715909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-q76rq"
Jul 22 15:18:47.985126 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:47.985087    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ltdzm"]
Jul 22 15:18:47.988539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:47.988513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:47.992076 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:47.991812    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wmqt8\""
Jul 22 15:18:47.999686 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:47.999648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ltdzm"]
Jul 22 15:18:48.044124 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.044084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6f5e492-7e55-45de-a5cf-03b745fd98be-bound-sa-token\") pod \"cert-manager-55b4d954d5-ltdzm\" (UID: \"a6f5e492-7e55-45de-a5cf-03b745fd98be\") " pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.044312 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.044219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7cvc\" (UniqueName: \"kubernetes.io/projected/a6f5e492-7e55-45de-a5cf-03b745fd98be-kube-api-access-t7cvc\") pod \"cert-manager-55b4d954d5-ltdzm\" (UID: \"a6f5e492-7e55-45de-a5cf-03b745fd98be\") " pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.145340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.145300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7cvc\" (UniqueName: \"kubernetes.io/projected/a6f5e492-7e55-45de-a5cf-03b745fd98be-kube-api-access-t7cvc\") pod \"cert-manager-55b4d954d5-ltdzm\" (UID: \"a6f5e492-7e55-45de-a5cf-03b745fd98be\") " pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.145340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.145346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6f5e492-7e55-45de-a5cf-03b745fd98be-bound-sa-token\") pod \"cert-manager-55b4d954d5-ltdzm\" (UID: \"a6f5e492-7e55-45de-a5cf-03b745fd98be\") " pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.155601 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.155569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6f5e492-7e55-45de-a5cf-03b745fd98be-bound-sa-token\") pod \"cert-manager-55b4d954d5-ltdzm\" (UID: \"a6f5e492-7e55-45de-a5cf-03b745fd98be\") " pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.155750 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.155699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7cvc\" (UniqueName: \"kubernetes.io/projected/a6f5e492-7e55-45de-a5cf-03b745fd98be-kube-api-access-t7cvc\") pod \"cert-manager-55b4d954d5-ltdzm\" (UID: \"a6f5e492-7e55-45de-a5cf-03b745fd98be\") " pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.302977 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.302938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ltdzm"
Jul 22 15:18:48.438849 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.438814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ltdzm"]
Jul 22 15:18:48.442074 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:18:48.442038    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6f5e492_7e55_45de_a5cf_03b745fd98be.slice/crio-165289b8bcfddb3a5b79407a53e4f79738857e715aa09d6feccb1f449fd4a2f5 WatchSource:0}: Error finding container 165289b8bcfddb3a5b79407a53e4f79738857e715aa09d6feccb1f449fd4a2f5: Status 404 returned error can't find the container with id 165289b8bcfddb3a5b79407a53e4f79738857e715aa09d6feccb1f449fd4a2f5
Jul 22 15:18:48.750287 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.750255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ltdzm" event={"ID":"a6f5e492-7e55-45de-a5cf-03b745fd98be","Type":"ContainerStarted","Data":"bb2b32a1176d270be77b62466500830d74e6145b09c55fbac8d33e5e883d2e3c"}
Jul 22 15:18:48.750476 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.750294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ltdzm" event={"ID":"a6f5e492-7e55-45de-a5cf-03b745fd98be","Type":"ContainerStarted","Data":"165289b8bcfddb3a5b79407a53e4f79738857e715aa09d6feccb1f449fd4a2f5"}
Jul 22 15:18:48.767971 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:18:48.767917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ltdzm" podStartSLOduration=1.76790033 podStartE2EDuration="1.76790033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:18:48.766569506 +0000 UTC m=+338.208652953" watchObservedRunningTime="2026-07-22 15:18:48.76790033 +0000 UTC m=+338.209983776"
Jul 22 15:19:10.879549 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.879513    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"]
Jul 22 15:19:10.882602 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.882576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:10.885189 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.885157    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 15:19:10.885626 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.885603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 15:19:10.885750 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.885643    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 15:19:10.885750 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.885690    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9pwxz\""
Jul 22 15:19:10.886167 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.886149    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 15:19:10.902310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:10.902279    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"]
Jul 22 15:19:11.051152 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.051123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zq6j\" (UniqueName: \"kubernetes.io/projected/e5a7131f-df0e-4c1b-9b89-c952ff423059-kube-api-access-5zq6j\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.051352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.051167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5a7131f-df0e-4c1b-9b89-c952ff423059-manifests\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.051352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.051200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5a7131f-df0e-4c1b-9b89-c952ff423059-tmp\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.051352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.051217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5a7131f-df0e-4c1b-9b89-c952ff423059-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.051352 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.051241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5a7131f-df0e-4c1b-9b89-c952ff423059-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.152188 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.152099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5a7131f-df0e-4c1b-9b89-c952ff423059-manifests\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.152188 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.152142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5a7131f-df0e-4c1b-9b89-c952ff423059-tmp\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.152188 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.152163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5a7131f-df0e-4c1b-9b89-c952ff423059-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.152188 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.152189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5a7131f-df0e-4c1b-9b89-c952ff423059-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.152464 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.152281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zq6j\" (UniqueName: \"kubernetes.io/projected/e5a7131f-df0e-4c1b-9b89-c952ff423059-kube-api-access-5zq6j\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.152666 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.152630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5a7131f-df0e-4c1b-9b89-c952ff423059-manifests\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.154922 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.154889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5a7131f-df0e-4c1b-9b89-c952ff423059-tmp\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.155050 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.154943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5a7131f-df0e-4c1b-9b89-c952ff423059-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.155050 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.154988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5a7131f-df0e-4c1b-9b89-c952ff423059-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.174634 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.174604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zq6j\" (UniqueName: \"kubernetes.io/projected/e5a7131f-df0e-4c1b-9b89-c952ff423059-kube-api-access-5zq6j\") pod \"opendatahub-operator-controller-manager-6c6f74d59b-zr6lg\" (UID: \"e5a7131f-df0e-4c1b-9b89-c952ff423059\") " pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.193570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.193538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:11.341276 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.341188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"]
Jul 22 15:19:11.349447 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:19:11.349419    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a7131f_df0e_4c1b_9b89_c952ff423059.slice/crio-094a72898597725fb68826e11424bc5af8fb1eba957d7d1ed6ddb3d7bbd86ccf WatchSource:0}: Error finding container 094a72898597725fb68826e11424bc5af8fb1eba957d7d1ed6ddb3d7bbd86ccf: Status 404 returned error can't find the container with id 094a72898597725fb68826e11424bc5af8fb1eba957d7d1ed6ddb3d7bbd86ccf
Jul 22 15:19:11.854111 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:11.854067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg" event={"ID":"e5a7131f-df0e-4c1b-9b89-c952ff423059","Type":"ContainerStarted","Data":"094a72898597725fb68826e11424bc5af8fb1eba957d7d1ed6ddb3d7bbd86ccf"}
Jul 22 15:19:14.873641 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:14.873604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg" event={"ID":"e5a7131f-df0e-4c1b-9b89-c952ff423059","Type":"ContainerStarted","Data":"91354fc66b54d6fe977fe0a67a7e2f368b455bc1674b2267952b64a433f7e850"}
Jul 22 15:19:15.879973 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:15.879940    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5a7131f-df0e-4c1b-9b89-c952ff423059" containerID="91354fc66b54d6fe977fe0a67a7e2f368b455bc1674b2267952b64a433f7e850" exitCode=0
Jul 22 15:19:15.880351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:15.880016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg" event={"ID":"e5a7131f-df0e-4c1b-9b89-c952ff423059","Type":"ContainerDied","Data":"91354fc66b54d6fe977fe0a67a7e2f368b455bc1674b2267952b64a433f7e850"}
Jul 22 15:19:16.888475 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:16.888434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg" event={"ID":"e5a7131f-df0e-4c1b-9b89-c952ff423059","Type":"ContainerStarted","Data":"7129657dd225c4702f0cf323401a550498f1ca1c7a1e9229614a51f017f64d8e"}
Jul 22 15:19:16.888967 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:16.888624    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:16.913459 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:16.913408    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg" podStartSLOduration=2.170438205 podStartE2EDuration="6.913389696s" totalImagesPullingTime="3.612885466s" totalInitContainerRuntime="1.130066025s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:19:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:19:16.912139374 +0000 UTC m=+366.354222821" watchObservedRunningTime="2026-07-22 15:19:16.913389696 +0000 UTC m=+366.355473143"
Jul 22 15:19:27.896413 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:27.896380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c6f74d59b-zr6lg"
Jul 22 15:19:28.324557 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.324516    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"]
Jul 22 15:19:28.340223 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.340184    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"]
Jul 22 15:19:28.340400 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.340305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.343230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.343202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8zgdh\""
Jul 22 15:19:28.343230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.343222    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 15:19:28.343230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.343232    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 15:19:28.343528 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.343230    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:19:28.343528 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.343202    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 15:19:28.343528 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.343349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 15:19:28.411139 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.411106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-manager-config\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.411340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.411196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-metrics-cert\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.411340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.411235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bwvv\" (UniqueName: \"kubernetes.io/projected/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-kube-api-access-5bwvv\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.411340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.411254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-cert\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.512403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.512366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-metrics-cert\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.512630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.512417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bwvv\" (UniqueName: \"kubernetes.io/projected/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-kube-api-access-5bwvv\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.512630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.512438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-cert\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.512630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.512467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-manager-config\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.513125 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.513106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-manager-config\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.515026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.515004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-cert\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.515104 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.515041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-metrics-cert\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.520827 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.520804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bwvv\" (UniqueName: \"kubernetes.io/projected/ace8cd7f-cffd-4144-b51f-0b65c82b2d21-kube-api-access-5bwvv\") pod \"lws-controller-manager-947d9d946-hrkk8\" (UID: \"ace8cd7f-cffd-4144-b51f-0b65c82b2d21\") " pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.651109 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.650990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:28.788974 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.788942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"]
Jul 22 15:19:28.792678 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:19:28.792651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace8cd7f_cffd_4144_b51f_0b65c82b2d21.slice/crio-c5885f718fec97de0caff84949d710426e50d38f646562a6c6811e69a6f7d23c WatchSource:0}: Error finding container c5885f718fec97de0caff84949d710426e50d38f646562a6c6811e69a6f7d23c: Status 404 returned error can't find the container with id c5885f718fec97de0caff84949d710426e50d38f646562a6c6811e69a6f7d23c
Jul 22 15:19:28.943136 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:28.943046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8" event={"ID":"ace8cd7f-cffd-4144-b51f-0b65c82b2d21","Type":"ContainerStarted","Data":"c5885f718fec97de0caff84949d710426e50d38f646562a6c6811e69a6f7d23c"}
Jul 22 15:19:31.961000 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:31.960956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8" event={"ID":"ace8cd7f-cffd-4144-b51f-0b65c82b2d21","Type":"ContainerStarted","Data":"baeb3fc8a78b6e8959a11ca19f6d95204cc845882ad693d9b0072364aad26503"}
Jul 22 15:19:31.961402 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:31.961090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:19:32.035592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.035517    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8" podStartSLOduration=1.622465195 podStartE2EDuration="4.035476993s" totalImagesPullingTime="2.413011798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:19:32.033848029 +0000 UTC m=+381.475931475" watchObservedRunningTime="2026-07-22 15:19:32.035476993 +0000 UTC m=+381.477560440"
Jul 22 15:19:32.215222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.215133    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"]
Jul 22 15:19:32.219276 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.219246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.222182 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.222161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 15:19:32.222952 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.222932    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 15:19:32.223814 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.223779    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 15:19:32.224086 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.224061    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 15:19:32.224346 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.224329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mzmp7\""
Jul 22 15:19:32.231480 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.231456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"]
Jul 22 15:19:32.248901 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.248872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj8wr\" (UniqueName: \"kubernetes.io/projected/4d117371-97e4-4744-9dcd-47b54f4e8050-kube-api-access-xj8wr\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.249067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.248918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d117371-97e4-4744-9dcd-47b54f4e8050-tmp\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.249067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.248945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d117371-97e4-4744-9dcd-47b54f4e8050-tls-certs\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.350153 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.350046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj8wr\" (UniqueName: \"kubernetes.io/projected/4d117371-97e4-4744-9dcd-47b54f4e8050-kube-api-access-xj8wr\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.350371 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.350188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d117371-97e4-4744-9dcd-47b54f4e8050-tmp\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.350371 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.350223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d117371-97e4-4744-9dcd-47b54f4e8050-tls-certs\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.350371 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:19:32.350325    2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 22 15:19:32.350527 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:19:32.350401    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d117371-97e4-4744-9dcd-47b54f4e8050-tls-certs podName:4d117371-97e4-4744-9dcd-47b54f4e8050 nodeName:}" failed. No retries permitted until 2026-07-22 15:19:32.850384054 +0000 UTC m=+382.292467479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4d117371-97e4-4744-9dcd-47b54f4e8050-tls-certs") pod "kube-auth-proxy-6764d75874-w9pwp" (UID: "4d117371-97e4-4744-9dcd-47b54f4e8050") : secret "kube-auth-proxy-tls" not found
Jul 22 15:19:32.352875 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.352840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d117371-97e4-4744-9dcd-47b54f4e8050-tmp\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.367168 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.367132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj8wr\" (UniqueName: \"kubernetes.io/projected/4d117371-97e4-4744-9dcd-47b54f4e8050-kube-api-access-xj8wr\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.855418 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.855380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d117371-97e4-4744-9dcd-47b54f4e8050-tls-certs\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:32.858644 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:32.858611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d117371-97e4-4744-9dcd-47b54f4e8050-tls-certs\") pod \"kube-auth-proxy-6764d75874-w9pwp\" (UID: \"4d117371-97e4-4744-9dcd-47b54f4e8050\") " pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:33.133007 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:33.132898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"
Jul 22 15:19:33.285428 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:33.285399    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6764d75874-w9pwp"]
Jul 22 15:19:33.287451 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:19:33.287425    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d117371_97e4_4744_9dcd_47b54f4e8050.slice/crio-1352ef24537fd9e1f49f3199a82a2dc26b79952758f7a969323850cc88063ec7 WatchSource:0}: Error finding container 1352ef24537fd9e1f49f3199a82a2dc26b79952758f7a969323850cc88063ec7: Status 404 returned error can't find the container with id 1352ef24537fd9e1f49f3199a82a2dc26b79952758f7a969323850cc88063ec7
Jul 22 15:19:33.972386 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:33.972347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp" event={"ID":"4d117371-97e4-4744-9dcd-47b54f4e8050","Type":"ContainerStarted","Data":"1352ef24537fd9e1f49f3199a82a2dc26b79952758f7a969323850cc88063ec7"}
Jul 22 15:19:36.992736 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:36.992698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp" event={"ID":"4d117371-97e4-4744-9dcd-47b54f4e8050","Type":"ContainerStarted","Data":"afc3bac285ec94d6ed67e45b0a27a949ea06ce26993c248878b5f48b11de5d84"}
Jul 22 15:19:37.010905 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:37.010843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6764d75874-w9pwp" podStartSLOduration=2.229808811 podStartE2EDuration="5.010826391s" totalImagesPullingTime="2.78101758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:19:37.009945939 +0000 UTC m=+386.452029415" watchObservedRunningTime="2026-07-22 15:19:37.010826391 +0000 UTC m=+386.452909838"
Jul 22 15:19:42.969248 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:19:42.969216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-947d9d946-hrkk8"
Jul 22 15:20:58.501686 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.501603    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b4b876cbd-zbgtb"]
Jul 22 15:20:58.506432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.506408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.510645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.510618    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 22 15:20:58.519072 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.519045    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 22 15:20:58.519276 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.519089    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z9lvv\""
Jul 22 15:20:58.519276 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.519227    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 22 15:20:58.519276 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.519237    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 22 15:20:58.519276 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.519254    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 22 15:20:58.526249 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4b876cbd-zbgtb"]
Jul 22 15:20:58.526673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-trusted-ca-bundle\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.526673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/830ee330-3845-447c-b2f6-2f71b12440b0-console-serving-cert\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.526903 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-oauth-serving-cert\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.526903 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8nhv\" (UniqueName: \"kubernetes.io/projected/830ee330-3845-447c-b2f6-2f71b12440b0-kube-api-access-t8nhv\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.526903 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-service-ca\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.527029 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.526905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-console-config\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.527066 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.527025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/830ee330-3845-447c-b2f6-2f71b12440b0-console-oauth-config\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.529732 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.529710    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 22 15:20:58.627725 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.627683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-service-ca\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.627725 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.627729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-console-config\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.627976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.627785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/830ee330-3845-447c-b2f6-2f71b12440b0-console-oauth-config\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.627976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.627803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-trusted-ca-bundle\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.627976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.627925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/830ee330-3845-447c-b2f6-2f71b12440b0-console-serving-cert\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.627976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.627967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-oauth-serving-cert\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.628174 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.628004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8nhv\" (UniqueName: \"kubernetes.io/projected/830ee330-3845-447c-b2f6-2f71b12440b0-kube-api-access-t8nhv\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.628578 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.628554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-console-config\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.628578 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.628565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-service-ca\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.628727 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.628717    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-oauth-serving-cert\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.629120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.629096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/830ee330-3845-447c-b2f6-2f71b12440b0-trusted-ca-bundle\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.630627 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.630606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/830ee330-3845-447c-b2f6-2f71b12440b0-console-oauth-config\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.630950 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.630934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/830ee330-3845-447c-b2f6-2f71b12440b0-console-serving-cert\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.636829 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.636805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8nhv\" (UniqueName: \"kubernetes.io/projected/830ee330-3845-447c-b2f6-2f71b12440b0-kube-api-access-t8nhv\") pod \"console-5b4b876cbd-zbgtb\" (UID: \"830ee330-3845-447c-b2f6-2f71b12440b0\") " pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.816207 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.816093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:20:58.959344 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:58.959317    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4b876cbd-zbgtb"]
Jul 22 15:20:58.962115 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:20:58.962085    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830ee330_3845_447c_b2f6_2f71b12440b0.slice/crio-c5bbc2110694f2c91e1f08582e906f85153f037272bd3497c9a7add00823c516 WatchSource:0}: Error finding container c5bbc2110694f2c91e1f08582e906f85153f037272bd3497c9a7add00823c516: Status 404 returned error can't find the container with id c5bbc2110694f2c91e1f08582e906f85153f037272bd3497c9a7add00823c516
Jul 22 15:20:59.363693 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:59.363651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4b876cbd-zbgtb" event={"ID":"830ee330-3845-447c-b2f6-2f71b12440b0","Type":"ContainerStarted","Data":"bd3acd500b578cb23ad8dc4a058ce18c7f7acfb2fe5ae0ce584b86ebd9fd8943"}
Jul 22 15:20:59.363893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:59.363713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4b876cbd-zbgtb" event={"ID":"830ee330-3845-447c-b2f6-2f71b12440b0","Type":"ContainerStarted","Data":"c5bbc2110694f2c91e1f08582e906f85153f037272bd3497c9a7add00823c516"}
Jul 22 15:20:59.382385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:20:59.382329    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b4b876cbd-zbgtb" podStartSLOduration=1.382310277 podStartE2EDuration="1.382310277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:20:59.381030508 +0000 UTC m=+468.823113956" watchObservedRunningTime="2026-07-22 15:20:59.382310277 +0000 UTC m=+468.824393724"
Jul 22 15:21:08.816966 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:08.816887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:21:08.817482 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:08.817049    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:21:08.822371 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:08.822339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:21:09.412935 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:09.412910    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b4b876cbd-zbgtb"
Jul 22 15:21:23.499239 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.499201    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"]
Jul 22 15:21:23.502782 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.502759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.505391 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.505368    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 15:21:23.505391 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.505377    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 15:21:23.505673 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.505660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9z9zk\""
Jul 22 15:21:23.506373 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.506356    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 15:21:23.506622 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.506606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 15:21:23.510660 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.510640    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"]
Jul 22 15:21:23.660932 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.660874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e810b721-be3d-4706-a0ac-c525477f9bb2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.661133 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.660945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5rqg\" (UniqueName: \"kubernetes.io/projected/e810b721-be3d-4706-a0ac-c525477f9bb2-kube-api-access-m5rqg\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.661133 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.661012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e810b721-be3d-4706-a0ac-c525477f9bb2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.762624 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.762477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e810b721-be3d-4706-a0ac-c525477f9bb2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.762624 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.762582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5rqg\" (UniqueName: \"kubernetes.io/projected/e810b721-be3d-4706-a0ac-c525477f9bb2-kube-api-access-m5rqg\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.762624 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.762627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e810b721-be3d-4706-a0ac-c525477f9bb2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.762898 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:21:23.762822    2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 22 15:21:23.762898 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:21:23.762895    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e810b721-be3d-4706-a0ac-c525477f9bb2-plugin-serving-cert podName:e810b721-be3d-4706-a0ac-c525477f9bb2 nodeName:}" failed. No retries permitted until 2026-07-22 15:21:24.262875531 +0000 UTC m=+493.704958956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e810b721-be3d-4706-a0ac-c525477f9bb2-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-x5bdp" (UID: "e810b721-be3d-4706-a0ac-c525477f9bb2") : secret "plugin-serving-cert" not found
Jul 22 15:21:23.763147 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.763129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e810b721-be3d-4706-a0ac-c525477f9bb2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:23.771906 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:23.771878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5rqg\" (UniqueName: \"kubernetes.io/projected/e810b721-be3d-4706-a0ac-c525477f9bb2-kube-api-access-m5rqg\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:24.269040 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:24.268998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e810b721-be3d-4706-a0ac-c525477f9bb2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:24.271675 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:24.271644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e810b721-be3d-4706-a0ac-c525477f9bb2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x5bdp\" (UID: \"e810b721-be3d-4706-a0ac-c525477f9bb2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:24.413621 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:24.413578    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"
Jul 22 15:21:24.554041 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:24.554015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp"]
Jul 22 15:21:24.556297 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:21:24.556267    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode810b721_be3d_4706_a0ac_c525477f9bb2.slice/crio-a22fdde7ac9d8ed185ab76a33a05cf0318c2302d0ed0194a2c1b35a7207b1a8b WatchSource:0}: Error finding container a22fdde7ac9d8ed185ab76a33a05cf0318c2302d0ed0194a2c1b35a7207b1a8b: Status 404 returned error can't find the container with id a22fdde7ac9d8ed185ab76a33a05cf0318c2302d0ed0194a2c1b35a7207b1a8b
Jul 22 15:21:25.491105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:25.491064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp" event={"ID":"e810b721-be3d-4706-a0ac-c525477f9bb2","Type":"ContainerStarted","Data":"a22fdde7ac9d8ed185ab76a33a05cf0318c2302d0ed0194a2c1b35a7207b1a8b"}
Jul 22 15:21:33.827907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.827869    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"]
Jul 22 15:21:33.846009 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.845576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:21:33.848253 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.848167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"]
Jul 22 15:21:33.848595 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.848355    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9fx7l\""
Jul 22 15:21:33.861250 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.861214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwfmt\" (UniqueName: \"kubernetes.io/projected/74b77eca-c773-43d7-bc37-40b6bb53fa6d-kube-api-access-nwfmt\") pod \"limitador-operator-controller-manager-859d7b78bd-qpwhc\" (UID: \"74b77eca-c773-43d7-bc37-40b6bb53fa6d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:21:33.961873 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.961834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwfmt\" (UniqueName: \"kubernetes.io/projected/74b77eca-c773-43d7-bc37-40b6bb53fa6d-kube-api-access-nwfmt\") pod \"limitador-operator-controller-manager-859d7b78bd-qpwhc\" (UID: \"74b77eca-c773-43d7-bc37-40b6bb53fa6d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:21:33.976093 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:33.976055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwfmt\" (UniqueName: \"kubernetes.io/projected/74b77eca-c773-43d7-bc37-40b6bb53fa6d-kube-api-access-nwfmt\") pod \"limitador-operator-controller-manager-859d7b78bd-qpwhc\" (UID: \"74b77eca-c773-43d7-bc37-40b6bb53fa6d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:21:34.160367 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:34.160265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:21:47.395376 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:47.395347    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"]
Jul 22 15:21:47.401342 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:21:47.401199    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b77eca_c773_43d7_bc37_40b6bb53fa6d.slice/crio-0eeb215cae1ee9de4b0517a633da77c604df42c139279605d5ba6e7bc1daaed5 WatchSource:0}: Error finding container 0eeb215cae1ee9de4b0517a633da77c604df42c139279605d5ba6e7bc1daaed5: Status 404 returned error can't find the container with id 0eeb215cae1ee9de4b0517a633da77c604df42c139279605d5ba6e7bc1daaed5
Jul 22 15:21:47.619369 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:47.619333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc" event={"ID":"74b77eca-c773-43d7-bc37-40b6bb53fa6d","Type":"ContainerStarted","Data":"0eeb215cae1ee9de4b0517a633da77c604df42c139279605d5ba6e7bc1daaed5"}
Jul 22 15:21:48.627605 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:48.627566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp" event={"ID":"e810b721-be3d-4706-a0ac-c525477f9bb2","Type":"ContainerStarted","Data":"eacc05de3b9faa4b711aef295984318b0953e5e4d73671de1d83202c63d0db85"}
Jul 22 15:21:48.643949 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:48.643858    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x5bdp" podStartSLOduration=2.384155508 podStartE2EDuration="25.643837737s" totalImagesPullingTime="23.259682229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:21:48.643109844 +0000 UTC m=+518.085193292" watchObservedRunningTime="2026-07-22 15:21:48.643837737 +0000 UTC m=+518.085921186"
Jul 22 15:21:49.635450 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:49.635352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc" event={"ID":"74b77eca-c773-43d7-bc37-40b6bb53fa6d","Type":"ContainerStarted","Data":"e95daa04617b06f3553b0a5359e79fd8b585a30e6f3a4e31ca5a8affd126f76a"}
Jul 22 15:21:49.652979 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:49.652912    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc" podStartSLOduration=14.986789496 podStartE2EDuration="16.652895871s" totalImagesPullingTime="1.666106375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:21:49.651883925 +0000 UTC m=+519.093967365" watchObservedRunningTime="2026-07-22 15:21:49.652895871 +0000 UTC m=+519.094979317"
Jul 22 15:21:50.640951 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:21:50.640920    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:22:01.648333 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:01.648251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qpwhc"
Jul 22 15:22:44.318173 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.318133    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-4tcpf"]
Jul 22 15:22:44.321603 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.321581    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.324093 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.324069    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:22:44.324972 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.324953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 15:22:44.325068 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.324984    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rrvsm\""
Jul 22 15:22:44.332094 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.332069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-4tcpf"]
Jul 22 15:22:44.436892 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.436842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7zfs\" (UniqueName: \"kubernetes.io/projected/844f8033-2f9e-47ec-b7df-19eaa19b42b3-kube-api-access-q7zfs\") pod \"maas-controller-79c87dd7d5-4tcpf\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") " pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.437090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.436910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/844f8033-2f9e-47ec-b7df-19eaa19b42b3-webhook-cert\") pod \"maas-controller-79c87dd7d5-4tcpf\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") " pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.537821 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.537781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7zfs\" (UniqueName: \"kubernetes.io/projected/844f8033-2f9e-47ec-b7df-19eaa19b42b3-kube-api-access-q7zfs\") pod \"maas-controller-79c87dd7d5-4tcpf\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") " pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.538015 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.537828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/844f8033-2f9e-47ec-b7df-19eaa19b42b3-webhook-cert\") pod \"maas-controller-79c87dd7d5-4tcpf\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") " pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.540564 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.540528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/844f8033-2f9e-47ec-b7df-19eaa19b42b3-webhook-cert\") pod \"maas-controller-79c87dd7d5-4tcpf\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") " pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.548224 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.548197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7zfs\" (UniqueName: \"kubernetes.io/projected/844f8033-2f9e-47ec-b7df-19eaa19b42b3-kube-api-access-q7zfs\") pod \"maas-controller-79c87dd7d5-4tcpf\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") " pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.632603 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.632509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:44.784144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.784116    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-4tcpf"]
Jul 22 15:22:44.787262 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:22:44.787235    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844f8033_2f9e_47ec_b7df_19eaa19b42b3.slice/crio-0d4d99586b533262383ddc68379c3e0565e09d512cb162f27e6863d2c7b83bd3 WatchSource:0}: Error finding container 0d4d99586b533262383ddc68379c3e0565e09d512cb162f27e6863d2c7b83bd3: Status 404 returned error can't find the container with id 0d4d99586b533262383ddc68379c3e0565e09d512cb162f27e6863d2c7b83bd3
Jul 22 15:22:44.904182 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:44.904096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf" event={"ID":"844f8033-2f9e-47ec-b7df-19eaa19b42b3","Type":"ContainerStarted","Data":"0d4d99586b533262383ddc68379c3e0565e09d512cb162f27e6863d2c7b83bd3"}
Jul 22 15:22:47.922807 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:47.922766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf" event={"ID":"844f8033-2f9e-47ec-b7df-19eaa19b42b3","Type":"ContainerStarted","Data":"5a95ad9db9bf9f8602b9992c772b709b6eb44a5907d899cdf24ac35055bf8fed"}
Jul 22 15:22:47.923203 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:47.922905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:22:47.963327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:47.963254    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf" podStartSLOduration=1.44168208 podStartE2EDuration="3.963230249s" totalImagesPullingTime="2.521548169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:22:47.954656923 +0000 UTC m=+577.396740369" watchObservedRunningTime="2026-07-22 15:22:47.963230249 +0000 UTC m=+577.405313697"
Jul 22 15:22:58.931027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:22:58.930994    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:23:20.495382 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.495347    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-nbb58"]
Jul 22 15:23:20.502104 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.502077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.504513 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.504478    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-f8k6w\""
Jul 22 15:23:20.504887 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.504864    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 15:23:20.513177 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.513146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-nbb58"]
Jul 22 15:23:20.585604 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.585566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8916e5f6-7c74-445b-919a-280c95aa3dea-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-nbb58\" (UID: \"8916e5f6-7c74-445b-919a-280c95aa3dea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.585793 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.585628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5lst\" (UniqueName: \"kubernetes.io/projected/8916e5f6-7c74-445b-919a-280c95aa3dea-kube-api-access-n5lst\") pod \"payload-pre-processing-7677b774bb-nbb58\" (UID: \"8916e5f6-7c74-445b-919a-280c95aa3dea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.686266 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.686209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8916e5f6-7c74-445b-919a-280c95aa3dea-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-nbb58\" (UID: \"8916e5f6-7c74-445b-919a-280c95aa3dea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.686455 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.686316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5lst\" (UniqueName: \"kubernetes.io/projected/8916e5f6-7c74-445b-919a-280c95aa3dea-kube-api-access-n5lst\") pod \"payload-pre-processing-7677b774bb-nbb58\" (UID: \"8916e5f6-7c74-445b-919a-280c95aa3dea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.687140 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.687115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8916e5f6-7c74-445b-919a-280c95aa3dea-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-nbb58\" (UID: \"8916e5f6-7c74-445b-919a-280c95aa3dea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.696438 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.696410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5lst\" (UniqueName: \"kubernetes.io/projected/8916e5f6-7c74-445b-919a-280c95aa3dea-kube-api-access-n5lst\") pod \"payload-pre-processing-7677b774bb-nbb58\" (UID: \"8916e5f6-7c74-445b-919a-280c95aa3dea\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.814071 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.814040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:20.974139 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:20.974111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-nbb58"]
Jul 22 15:23:20.975369 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:23:20.975337    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8916e5f6_7c74_445b_919a_280c95aa3dea.slice/crio-c1a8593187ed32b0fda2e0a28043e4e0f5def70415f9fb7d4f4bffbeb18390e0 WatchSource:0}: Error finding container c1a8593187ed32b0fda2e0a28043e4e0f5def70415f9fb7d4f4bffbeb18390e0: Status 404 returned error can't find the container with id c1a8593187ed32b0fda2e0a28043e4e0f5def70415f9fb7d4f4bffbeb18390e0
Jul 22 15:23:21.087198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:21.087104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58" event={"ID":"8916e5f6-7c74-445b-919a-280c95aa3dea","Type":"ContainerStarted","Data":"c1a8593187ed32b0fda2e0a28043e4e0f5def70415f9fb7d4f4bffbeb18390e0"}
Jul 22 15:23:23.098925 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:23.098876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58" event={"ID":"8916e5f6-7c74-445b-919a-280c95aa3dea","Type":"ContainerStarted","Data":"a2d59a3db132ad71e6495d337d29ac4e13889c4ecc6acea3f8d077ecaac1068f"}
Jul 22 15:23:23.099336 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:23.099025    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:23:23.118716 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:23.118661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58" podStartSLOduration=1.4478918969999999 podStartE2EDuration="3.118637003s" totalImagesPullingTime="1.670745106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:23:23.117046374 +0000 UTC m=+612.559129820" watchObservedRunningTime="2026-07-22 15:23:23.118637003 +0000 UTC m=+612.560720448"
Jul 22 15:23:34.105329 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:23:34.105285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-nbb58"
Jul 22 15:24:05.420451 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.420408    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"]
Jul 22 15:24:05.424114 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.424090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.426698 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.426673    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xftm8\""
Jul 22 15:24:05.427610 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.427586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 15:24:05.427610 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.427604    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 22 15:24:05.427785 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.427642    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 15:24:05.434989 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.434961    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"]
Jul 22 15:24:05.510965 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.510921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a62971de-6217-4fda-89a9-ad453c3235aa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.510965 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.510966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.511186 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.511080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2mtr\" (UniqueName: \"kubernetes.io/projected/a62971de-6217-4fda-89a9-ad453c3235aa-kube-api-access-w2mtr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.511186 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.511118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.511186 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.511143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.511186 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.511161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a62971de-6217-4fda-89a9-ad453c3235aa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612479 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612479 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2mtr\" (UniqueName: \"kubernetes.io/projected/a62971de-6217-4fda-89a9-ad453c3235aa-kube-api-access-w2mtr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612479 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612653 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612799 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.612991 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.612970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.613074 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.613023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.613150 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.613128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.614927 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.614904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a62971de-6217-4fda-89a9-ad453c3235aa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.615155 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.615138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a62971de-6217-4fda-89a9-ad453c3235aa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.620216 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.620192    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2mtr\" (UniqueName: \"kubernetes.io/projected/a62971de-6217-4fda-89a9-ad453c3235aa-kube-api-access-w2mtr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-789tr\" (UID: \"a62971de-6217-4fda-89a9-ad453c3235aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.736444 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.736337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:05.879368 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.879338    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"]
Jul 22 15:24:05.881536 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:24:05.881502    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62971de_6217_4fda_89a9_ad453c3235aa.slice/crio-2db57cd5254e2ea3e7b64ac188e212a0958b1d4edcf47adae38e080fd1b57e43 WatchSource:0}: Error finding container 2db57cd5254e2ea3e7b64ac188e212a0958b1d4edcf47adae38e080fd1b57e43: Status 404 returned error can't find the container with id 2db57cd5254e2ea3e7b64ac188e212a0958b1d4edcf47adae38e080fd1b57e43
Jul 22 15:24:05.882919 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:05.882901    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:24:06.307460 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:06.307417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr" event={"ID":"a62971de-6217-4fda-89a9-ad453c3235aa","Type":"ContainerStarted","Data":"2db57cd5254e2ea3e7b64ac188e212a0958b1d4edcf47adae38e080fd1b57e43"}
Jul 22 15:24:07.313685 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:07.313646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr" event={"ID":"a62971de-6217-4fda-89a9-ad453c3235aa","Type":"ContainerStarted","Data":"9bacc9c7de1fcac76fd61946c26da6b6bd13286822ee048cdac708add87bf5d8"}
Jul 22 15:24:07.332741 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:07.332686    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr" podStartSLOduration=1.114566497 podStartE2EDuration="2.332666036s" totalImagesPullingTime="1.218099539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:24:07.331665569 +0000 UTC m=+656.773749017" watchObservedRunningTime="2026-07-22 15:24:07.332666036 +0000 UTC m=+656.774749484"
Jul 22 15:24:15.736658 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:15.736608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:15.737139 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:15.736751    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:15.749400 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:15.749371    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:16.376160 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:16.376121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-789tr"
Jul 22 15:24:52.222909 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.222865    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"]
Jul 22 15:24:52.226381 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.226355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.229074 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.229047    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 22 15:24:52.235386 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.235356    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"]
Jul 22 15:24:52.347184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.347142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.347370 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.347195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.347370 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.347223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.347370 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.347275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.347517 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.347366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a655895-af42-4a51-befb-3c6ba9e8dbd7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.347517 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.347401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnwjl\" (UniqueName: \"kubernetes.io/projected/1a655895-af42-4a51-befb-3c6ba9e8dbd7-kube-api-access-fnwjl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.448718 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.448677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnwjl\" (UniqueName: \"kubernetes.io/projected/1a655895-af42-4a51-befb-3c6ba9e8dbd7-kube-api-access-fnwjl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.448897 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.448732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.448897 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.448774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.448897 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.448796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.448897 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.448851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.449052 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.448933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a655895-af42-4a51-befb-3c6ba9e8dbd7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.449300 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.449261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.449300 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.449299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.449458 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.449341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.451432 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.451402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a655895-af42-4a51-befb-3c6ba9e8dbd7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.451759 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.451738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a655895-af42-4a51-befb-3c6ba9e8dbd7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.457336 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.457304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnwjl\" (UniqueName: \"kubernetes.io/projected/1a655895-af42-4a51-befb-3c6ba9e8dbd7-kube-api-access-fnwjl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4vl84\" (UID: \"1a655895-af42-4a51-befb-3c6ba9e8dbd7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.537353 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.537325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:24:52.686647 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:52.686620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"]
Jul 22 15:24:52.688349 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:24:52.688320    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a655895_af42_4a51_befb_3c6ba9e8dbd7.slice/crio-998fa38574545846dd89273d48919031caeff6d246cd16b841caa6718fdc890d WatchSource:0}: Error finding container 998fa38574545846dd89273d48919031caeff6d246cd16b841caa6718fdc890d: Status 404 returned error can't find the container with id 998fa38574545846dd89273d48919031caeff6d246cd16b841caa6718fdc890d
Jul 22 15:24:53.546821 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:53.546787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84" event={"ID":"1a655895-af42-4a51-befb-3c6ba9e8dbd7","Type":"ContainerStarted","Data":"998fa38574545846dd89273d48919031caeff6d246cd16b841caa6718fdc890d"}
Jul 22 15:24:56.563160 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:56.563121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84" event={"ID":"1a655895-af42-4a51-befb-3c6ba9e8dbd7","Type":"ContainerStarted","Data":"737dcdb29486442b688358b1e2a6c33049b864f17d1b6c08ced63b07bfce274f"}
Jul 22 15:24:56.580123 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:24:56.580068    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84" podStartSLOduration=1.219115223 podStartE2EDuration="4.580053054s" totalImagesPullingTime="3.360937831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:24:56.579412191 +0000 UTC m=+706.021495639" watchObservedRunningTime="2026-07-22 15:24:56.580053054 +0000 UTC m=+706.022136501"
Jul 22 15:25:02.538165 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:25:02.538062    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:25:02.538165 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:25:02.538116    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:25:02.550897 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:25:02.550861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:25:02.604125 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:25:02.604086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4vl84"
Jul 22 15:27:04.021580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.020956    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-4tcpf"]
Jul 22 15:27:04.021580 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.021342    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf" podUID="844f8033-2f9e-47ec-b7df-19eaa19b42b3" containerName="manager" containerID="cri-o://5a95ad9db9bf9f8602b9992c772b709b6eb44a5907d899cdf24ac35055bf8fed" gracePeriod=30
Jul 22 15:27:04.198867 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.198833    2580 generic.go:356] "Generic (PLEG): container finished" podID="844f8033-2f9e-47ec-b7df-19eaa19b42b3" containerID="5a95ad9db9bf9f8602b9992c772b709b6eb44a5907d899cdf24ac35055bf8fed" exitCode=0
Jul 22 15:27:04.199067 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.198905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf" event={"ID":"844f8033-2f9e-47ec-b7df-19eaa19b42b3","Type":"ContainerDied","Data":"5a95ad9db9bf9f8602b9992c772b709b6eb44a5907d899cdf24ac35055bf8fed"}
Jul 22 15:27:04.284775 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.284704    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:27:04.430291 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.430256    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/844f8033-2f9e-47ec-b7df-19eaa19b42b3-webhook-cert\") pod \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") "
Jul 22 15:27:04.430473 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.430369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7zfs\" (UniqueName: \"kubernetes.io/projected/844f8033-2f9e-47ec-b7df-19eaa19b42b3-kube-api-access-q7zfs\") pod \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\" (UID: \"844f8033-2f9e-47ec-b7df-19eaa19b42b3\") "
Jul 22 15:27:04.432736 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.432696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/844f8033-2f9e-47ec-b7df-19eaa19b42b3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "844f8033-2f9e-47ec-b7df-19eaa19b42b3" (UID: "844f8033-2f9e-47ec-b7df-19eaa19b42b3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:27:04.432868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.432741    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844f8033-2f9e-47ec-b7df-19eaa19b42b3-kube-api-access-q7zfs" (OuterVolumeSpecName: "kube-api-access-q7zfs") pod "844f8033-2f9e-47ec-b7df-19eaa19b42b3" (UID: "844f8033-2f9e-47ec-b7df-19eaa19b42b3"). InnerVolumeSpecName "kube-api-access-q7zfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:27:04.531297 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.531256    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7zfs\" (UniqueName: \"kubernetes.io/projected/844f8033-2f9e-47ec-b7df-19eaa19b42b3-kube-api-access-q7zfs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:27:04.531297 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:04.531290    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/844f8033-2f9e-47ec-b7df-19eaa19b42b3-webhook-cert\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:27:05.205834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:05.205804    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf"
Jul 22 15:27:05.205834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:05.205824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-4tcpf" event={"ID":"844f8033-2f9e-47ec-b7df-19eaa19b42b3","Type":"ContainerDied","Data":"0d4d99586b533262383ddc68379c3e0565e09d512cb162f27e6863d2c7b83bd3"}
Jul 22 15:27:05.206349 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:05.205893    2580 scope.go:117] "RemoveContainer" containerID="5a95ad9db9bf9f8602b9992c772b709b6eb44a5907d899cdf24ac35055bf8fed"
Jul 22 15:27:05.224743 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:05.224700    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-4tcpf"]
Jul 22 15:27:05.227833 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:05.227796    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-4tcpf"]
Jul 22 15:27:07.022266 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:27:07.022228    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="844f8033-2f9e-47ec-b7df-19eaa19b42b3" path="/var/lib/kubelet/pods/844f8033-2f9e-47ec-b7df-19eaa19b42b3/volumes"
Jul 22 15:28:06.097372 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.097284    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-x8c5p"]
Jul 22 15:28:06.097893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.097705    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844f8033-2f9e-47ec-b7df-19eaa19b42b3" containerName="manager"
Jul 22 15:28:06.097893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.097717    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="844f8033-2f9e-47ec-b7df-19eaa19b42b3" containerName="manager"
Jul 22 15:28:06.097893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.097784    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="844f8033-2f9e-47ec-b7df-19eaa19b42b3" containerName="manager"
Jul 22 15:28:06.101064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.101041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.103843 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.103819    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rrvsm\""
Jul 22 15:28:06.104656 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.104637    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 15:28:06.104706 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.104664    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:28:06.112025 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.111989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-x8c5p"]
Jul 22 15:28:06.123254 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.123217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5jzr\" (UniqueName: \"kubernetes.io/projected/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-kube-api-access-b5jzr\") pod \"maas-controller-79c87dd7d5-x8c5p\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") " pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.123536 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.123473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-webhook-cert\") pod \"maas-controller-79c87dd7d5-x8c5p\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") " pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.224312 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.224269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-webhook-cert\") pod \"maas-controller-79c87dd7d5-x8c5p\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") " pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.224522 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.224358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5jzr\" (UniqueName: \"kubernetes.io/projected/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-kube-api-access-b5jzr\") pod \"maas-controller-79c87dd7d5-x8c5p\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") " pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.227050 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.227021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-webhook-cert\") pod \"maas-controller-79c87dd7d5-x8c5p\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") " pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.235407 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.235379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5jzr\" (UniqueName: \"kubernetes.io/projected/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-kube-api-access-b5jzr\") pod \"maas-controller-79c87dd7d5-x8c5p\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") " pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.431907 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.431796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:06.582639 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:06.582603    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-x8c5p"]
Jul 22 15:28:06.585902 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:28:06.585873    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c74df5d_ff1f_43fc_a844_2402fca6bdc1.slice/crio-9ee6fabff429c0f64fc45f15ab8b88f8c5f155ed5dd6ffa6d8196467238b3d40 WatchSource:0}: Error finding container 9ee6fabff429c0f64fc45f15ab8b88f8c5f155ed5dd6ffa6d8196467238b3d40: Status 404 returned error can't find the container with id 9ee6fabff429c0f64fc45f15ab8b88f8c5f155ed5dd6ffa6d8196467238b3d40
Jul 22 15:28:07.514190 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:07.514151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p" event={"ID":"7c74df5d-ff1f-43fc-a844-2402fca6bdc1","Type":"ContainerStarted","Data":"cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3"}
Jul 22 15:28:07.514633 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:07.514329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:07.514633 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:07.514374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p" event={"ID":"7c74df5d-ff1f-43fc-a844-2402fca6bdc1","Type":"ContainerStarted","Data":"9ee6fabff429c0f64fc45f15ab8b88f8c5f155ed5dd6ffa6d8196467238b3d40"}
Jul 22 15:28:07.534221 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:07.534159    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p" podStartSLOduration=1.024453646 podStartE2EDuration="1.534139494s" totalImagesPullingTime="509.685848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:28:07.532595559 +0000 UTC m=+896.974679007" watchObservedRunningTime="2026-07-22 15:28:07.534139494 +0000 UTC m=+896.976222940"
Jul 22 15:28:18.522120 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:18.522088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:23.188152 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.188115    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-x8c5p"]
Jul 22 15:28:23.188682 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.188341    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p" podUID="7c74df5d-ff1f-43fc-a844-2402fca6bdc1" containerName="manager" containerID="cri-o://cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3" gracePeriod=30
Jul 22 15:28:23.445279 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.445214    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:23.487198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.487166    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-webhook-cert\") pod \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") "
Jul 22 15:28:23.487198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.487204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5jzr\" (UniqueName: \"kubernetes.io/projected/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-kube-api-access-b5jzr\") pod \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\" (UID: \"7c74df5d-ff1f-43fc-a844-2402fca6bdc1\") "
Jul 22 15:28:23.489433 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.489399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7c74df5d-ff1f-43fc-a844-2402fca6bdc1" (UID: "7c74df5d-ff1f-43fc-a844-2402fca6bdc1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:28:23.489577 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.489482    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-kube-api-access-b5jzr" (OuterVolumeSpecName: "kube-api-access-b5jzr") pod "7c74df5d-ff1f-43fc-a844-2402fca6bdc1" (UID: "7c74df5d-ff1f-43fc-a844-2402fca6bdc1"). InnerVolumeSpecName "kube-api-access-b5jzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:28:23.588345 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.588308    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-webhook-cert\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:28:23.588345 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.588340    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5jzr\" (UniqueName: \"kubernetes.io/projected/7c74df5d-ff1f-43fc-a844-2402fca6bdc1-kube-api-access-b5jzr\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:28:23.597544 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.597484    2580 generic.go:356] "Generic (PLEG): container finished" podID="7c74df5d-ff1f-43fc-a844-2402fca6bdc1" containerID="cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3" exitCode=0
Jul 22 15:28:23.597722 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.597560    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p"
Jul 22 15:28:23.597722 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.597582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p" event={"ID":"7c74df5d-ff1f-43fc-a844-2402fca6bdc1","Type":"ContainerDied","Data":"cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3"}
Jul 22 15:28:23.597722 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.597637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-x8c5p" event={"ID":"7c74df5d-ff1f-43fc-a844-2402fca6bdc1","Type":"ContainerDied","Data":"9ee6fabff429c0f64fc45f15ab8b88f8c5f155ed5dd6ffa6d8196467238b3d40"}
Jul 22 15:28:23.597722 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.597669    2580 scope.go:117] "RemoveContainer" containerID="cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3"
Jul 22 15:28:23.609655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.609638    2580 scope.go:117] "RemoveContainer" containerID="cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3"
Jul 22 15:28:23.609965 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:28:23.609936    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3\": container with ID starting with cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3 not found: ID does not exist" containerID="cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3"
Jul 22 15:28:23.610015 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.609973    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3"} err="failed to get container status \"cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3\": rpc error: code = NotFound desc = could not find container \"cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3\": container with ID starting with cb6b16466a28dd6f507c324b6326f0f00a2a6da472f0676716a817eb3a025cc3 not found: ID does not exist"
Jul 22 15:28:23.621280 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.621244    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-x8c5p"]
Jul 22 15:28:23.625048 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:23.625023    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-x8c5p"]
Jul 22 15:28:24.440865 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.440827    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-qtl5r"]
Jul 22 15:28:24.441244 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.441201    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c74df5d-ff1f-43fc-a844-2402fca6bdc1" containerName="manager"
Jul 22 15:28:24.441244 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.441212    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c74df5d-ff1f-43fc-a844-2402fca6bdc1" containerName="manager"
Jul 22 15:28:24.441313 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.441296    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c74df5d-ff1f-43fc-a844-2402fca6bdc1" containerName="manager"
Jul 22 15:28:24.451435 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.451401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.454021 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.453995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-qtl5r"]
Jul 22 15:28:24.455388 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.455364    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:28:24.455541 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.455388    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 15:28:24.455541 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.455391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rrvsm\""
Jul 22 15:28:24.496238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.496200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztlwz\" (UniqueName: \"kubernetes.io/projected/8b5239a3-0add-4c40-9a24-9ae1843bc279-kube-api-access-ztlwz\") pod \"maas-controller-79c87dd7d5-qtl5r\" (UID: \"8b5239a3-0add-4c40-9a24-9ae1843bc279\") " pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.496408 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.496284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b5239a3-0add-4c40-9a24-9ae1843bc279-webhook-cert\") pod \"maas-controller-79c87dd7d5-qtl5r\" (UID: \"8b5239a3-0add-4c40-9a24-9ae1843bc279\") " pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.596766 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.596733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztlwz\" (UniqueName: \"kubernetes.io/projected/8b5239a3-0add-4c40-9a24-9ae1843bc279-kube-api-access-ztlwz\") pod \"maas-controller-79c87dd7d5-qtl5r\" (UID: \"8b5239a3-0add-4c40-9a24-9ae1843bc279\") " pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.596943 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.596791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b5239a3-0add-4c40-9a24-9ae1843bc279-webhook-cert\") pod \"maas-controller-79c87dd7d5-qtl5r\" (UID: \"8b5239a3-0add-4c40-9a24-9ae1843bc279\") " pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.599661 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.599632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b5239a3-0add-4c40-9a24-9ae1843bc279-webhook-cert\") pod \"maas-controller-79c87dd7d5-qtl5r\" (UID: \"8b5239a3-0add-4c40-9a24-9ae1843bc279\") " pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.605288 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.605259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztlwz\" (UniqueName: \"kubernetes.io/projected/8b5239a3-0add-4c40-9a24-9ae1843bc279-kube-api-access-ztlwz\") pod \"maas-controller-79c87dd7d5-qtl5r\" (UID: \"8b5239a3-0add-4c40-9a24-9ae1843bc279\") " pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.765548 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.765485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:24.909777 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:24.909756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c87dd7d5-qtl5r"]
Jul 22 15:28:24.912301 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:28:24.912272    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5239a3_0add_4c40_9a24_9ae1843bc279.slice/crio-16b442e4e33d60ed110bf6bfcb509580131ccbb1b36ab704a2baa20cbb38f634 WatchSource:0}: Error finding container 16b442e4e33d60ed110bf6bfcb509580131ccbb1b36ab704a2baa20cbb38f634: Status 404 returned error can't find the container with id 16b442e4e33d60ed110bf6bfcb509580131ccbb1b36ab704a2baa20cbb38f634
Jul 22 15:28:25.021161 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:25.021079    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c74df5d-ff1f-43fc-a844-2402fca6bdc1" path="/var/lib/kubelet/pods/7c74df5d-ff1f-43fc-a844-2402fca6bdc1/volumes"
Jul 22 15:28:25.612235 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:25.612202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r" event={"ID":"8b5239a3-0add-4c40-9a24-9ae1843bc279","Type":"ContainerStarted","Data":"0dd01c8a8e4079013739e0aa85dd4a2512cd22684e41d4a1216a8b72bc702052"}
Jul 22 15:28:25.612729 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:25.612257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r" event={"ID":"8b5239a3-0add-4c40-9a24-9ae1843bc279","Type":"ContainerStarted","Data":"16b442e4e33d60ed110bf6bfcb509580131ccbb1b36ab704a2baa20cbb38f634"}
Jul 22 15:28:25.612729 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:25.612293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:28:25.633610 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:25.633559    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r" podStartSLOduration=1.059850161 podStartE2EDuration="1.633542023s" totalImagesPullingTime="573.691862ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:28:25.63228393 +0000 UTC m=+915.074367377" watchObservedRunningTime="2026-07-22 15:28:25.633542023 +0000 UTC m=+915.075625470"
Jul 22 15:28:36.620040 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:28:36.620010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79c87dd7d5-qtl5r"
Jul 22 15:30:00.148314 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.148228    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"]
Jul 22 15:30:00.151933 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.151911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"
Jul 22 15:30:00.154281 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.154260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:30:00.157756 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.157728    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"]
Jul 22 15:30:00.163711 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.163688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"
Jul 22 15:30:00.310014 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.309990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"]
Jul 22 15:30:00.312188 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:30:00.312156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d9bbb8_8a32_4082_a348_59df96dd159d.slice/crio-224df1ddf6b24aead873e04959262c445f5939806156db52960ab78e11c8348c WatchSource:0}: Error finding container 224df1ddf6b24aead873e04959262c445f5939806156db52960ab78e11c8348c: Status 404 returned error can't find the container with id 224df1ddf6b24aead873e04959262c445f5939806156db52960ab78e11c8348c
Jul 22 15:30:00.314203 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:00.314185    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:30:01.091050 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:01.091017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4" event={"ID":"06d9bbb8-8a32-4082-a348-59df96dd159d","Type":"ContainerStarted","Data":"224df1ddf6b24aead873e04959262c445f5939806156db52960ab78e11c8348c"}
Jul 22 15:30:04.108735 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:04.108700    2580 generic.go:356] "Generic (PLEG): container finished" podID="06d9bbb8-8a32-4082-a348-59df96dd159d" containerID="1a3b47b61b2d28d0a5ae20931d91bfe5435be0138e529e769c678cc61d5f34fe" exitCode=0
Jul 22 15:30:04.109137 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:04.108743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4" event={"ID":"06d9bbb8-8a32-4082-a348-59df96dd159d","Type":"ContainerDied","Data":"1a3b47b61b2d28d0a5ae20931d91bfe5435be0138e529e769c678cc61d5f34fe"}
Jul 22 15:30:05.251546 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:05.251514    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"
Jul 22 15:30:06.120438 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:06.120398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4" event={"ID":"06d9bbb8-8a32-4082-a348-59df96dd159d","Type":"ContainerDied","Data":"224df1ddf6b24aead873e04959262c445f5939806156db52960ab78e11c8348c"}
Jul 22 15:30:06.120438 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:06.120435    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-lfld4"
Jul 22 15:30:06.120438 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:30:06.120449    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="224df1ddf6b24aead873e04959262c445f5939806156db52960ab78e11c8348c"
Jul 22 15:42:06.742989 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.742952    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"]
Jul 22 15:42:06.743575 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.743557    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d9bbb8-8a32-4082-a348-59df96dd159d" containerName="cleanup"
Jul 22 15:42:06.743640 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.743579    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d9bbb8-8a32-4082-a348-59df96dd159d" containerName="cleanup"
Jul 22 15:42:06.743726 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.743713    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d9bbb8-8a32-4082-a348-59df96dd159d" containerName="cleanup"
Jul 22 15:42:06.750459 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.748507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:06.754303 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.754272    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:42:06.755177 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.754285    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:42:06.756798 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.756770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"]
Jul 22 15:42:06.814611 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.814576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-88eef265-m6nlc\" (UID: \"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:06.915410 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.915374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-88eef265-m6nlc\" (UID: \"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:06.916043 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:06.916014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-88eef265-m6nlc\" (UID: \"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:07.064241 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:07.064205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:07.416590 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:07.416566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"]
Jul 22 15:42:07.419108 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:42:07.419076    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38344b3a_39b4_4e0f_aa67_dcfa58b74ea1.slice/crio-fe31cba1783c3c4b97ce6978de1e9abcce99f6a29e5f1939d00270d2fdd2d948 WatchSource:0}: Error finding container fe31cba1783c3c4b97ce6978de1e9abcce99f6a29e5f1939d00270d2fdd2d948: Status 404 returned error can't find the container with id fe31cba1783c3c4b97ce6978de1e9abcce99f6a29e5f1939d00270d2fdd2d948
Jul 22 15:42:07.692630 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:07.692545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc" event={"ID":"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1","Type":"ContainerStarted","Data":"fe31cba1783c3c4b97ce6978de1e9abcce99f6a29e5f1939d00270d2fdd2d948"}
Jul 22 15:42:08.699549 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:08.699520    2580 generic.go:356] "Generic (PLEG): container finished" podID="38344b3a-39b4-4e0f-aa67-dcfa58b74ea1" containerID="e06418e740daadcb4f767aabf712e582ff144de4be851977e46c94ca3ebba57d" exitCode=0
Jul 22 15:42:08.699958 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:08.699606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc" event={"ID":"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1","Type":"ContainerDied","Data":"e06418e740daadcb4f767aabf712e582ff144de4be851977e46c94ca3ebba57d"}
Jul 22 15:42:09.839307 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:09.839281    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:09.944336 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:09.944303    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1-maas-api-service-ca\") pod \"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1\" (UID: \"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1\") "
Jul 22 15:42:09.944693 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:09.944668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38344b3a-39b4-4e0f-aa67-dcfa58b74ea1" (UID: "38344b3a-39b4-4e0f-aa67-dcfa58b74ea1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:42:10.046594 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:10.046547    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:42:10.710465 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:10.710436    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"
Jul 22 15:42:10.710648 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:10.710434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc" event={"ID":"38344b3a-39b4-4e0f-aa67-dcfa58b74ea1","Type":"ContainerDied","Data":"fe31cba1783c3c4b97ce6978de1e9abcce99f6a29e5f1939d00270d2fdd2d948"}
Jul 22 15:42:10.710648 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:10.710535    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe31cba1783c3c4b97ce6978de1e9abcce99f6a29e5f1939d00270d2fdd2d948"
Jul 22 15:42:21.821512 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:21.821457    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"]
Jul 22 15:42:21.823215 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:21.823194    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-88eef265-m6nlc"]
Jul 22 15:42:23.021095 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:23.021064    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38344b3a-39b4-4e0f-aa67-dcfa58b74ea1" path="/var/lib/kubelet/pods/38344b3a-39b4-4e0f-aa67-dcfa58b74ea1/volumes"
Jul 22 15:42:50.469836 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.469797    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"]
Jul 22 15:42:50.470388 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.470328    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38344b3a-39b4-4e0f-aa67-dcfa58b74ea1" containerName="revoke-keys"
Jul 22 15:42:50.470388 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.470344    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="38344b3a-39b4-4e0f-aa67-dcfa58b74ea1" containerName="revoke-keys"
Jul 22 15:42:50.470518 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.470416    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="38344b3a-39b4-4e0f-aa67-dcfa58b74ea1" containerName="revoke-keys"
Jul 22 15:42:50.474440 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.474411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:42:50.478117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.478094    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:42:50.478611 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.478593    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:42:50.482999 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.482971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"]
Jul 22 15:42:50.528571 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.528532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/605a8ceb-a25e-4a9a-8235-fd10d848d57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg\" (UID: \"605a8ceb-a25e-4a9a-8235-fd10d848d57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:42:50.629254 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.629209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/605a8ceb-a25e-4a9a-8235-fd10d848d57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg\" (UID: \"605a8ceb-a25e-4a9a-8235-fd10d848d57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:42:50.629890 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.629872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/605a8ceb-a25e-4a9a-8235-fd10d848d57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg\" (UID: \"605a8ceb-a25e-4a9a-8235-fd10d848d57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:42:50.790338 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.790301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:42:50.930286 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:50.930260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"]
Jul 22 15:42:50.933185 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:42:50.933158    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605a8ceb_a25e_4a9a_8235_fd10d848d57e.slice/crio-ac4511e706b433c9b20c1b6c6922300a572d5609a12d8a95d1c74c3036fb1230 WatchSource:0}: Error finding container ac4511e706b433c9b20c1b6c6922300a572d5609a12d8a95d1c74c3036fb1230: Status 404 returned error can't find the container with id ac4511e706b433c9b20c1b6c6922300a572d5609a12d8a95d1c74c3036fb1230
Jul 22 15:42:51.929648 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:51.929551    2580 generic.go:356] "Generic (PLEG): container finished" podID="605a8ceb-a25e-4a9a-8235-fd10d848d57e" containerID="bb5c4c91dd2ab65a60d05b9d92e182ff1829f324b3c20415f45d9ad7edcf60e3" exitCode=0
Jul 22 15:42:51.929648 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:51.929605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg" event={"ID":"605a8ceb-a25e-4a9a-8235-fd10d848d57e","Type":"ContainerDied","Data":"bb5c4c91dd2ab65a60d05b9d92e182ff1829f324b3c20415f45d9ad7edcf60e3"}
Jul 22 15:42:51.930061 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:51.929667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg" event={"ID":"605a8ceb-a25e-4a9a-8235-fd10d848d57e","Type":"ContainerStarted","Data":"ac4511e706b433c9b20c1b6c6922300a572d5609a12d8a95d1c74c3036fb1230"}
Jul 22 15:42:53.072842 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.072818    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:42:53.151877 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.151845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/605a8ceb-a25e-4a9a-8235-fd10d848d57e-maas-api-service-ca\") pod \"605a8ceb-a25e-4a9a-8235-fd10d848d57e\" (UID: \"605a8ceb-a25e-4a9a-8235-fd10d848d57e\") "
Jul 22 15:42:53.152251 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.152226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/605a8ceb-a25e-4a9a-8235-fd10d848d57e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "605a8ceb-a25e-4a9a-8235-fd10d848d57e" (UID: "605a8ceb-a25e-4a9a-8235-fd10d848d57e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:42:53.253170 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.253141    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/605a8ceb-a25e-4a9a-8235-fd10d848d57e-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:42:53.940809 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.940768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg" event={"ID":"605a8ceb-a25e-4a9a-8235-fd10d848d57e","Type":"ContainerDied","Data":"ac4511e706b433c9b20c1b6c6922300a572d5609a12d8a95d1c74c3036fb1230"}
Jul 22 15:42:53.941020 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.940829    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac4511e706b433c9b20c1b6c6922300a572d5609a12d8a95d1c74c3036fb1230"
Jul 22 15:42:53.941020 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:42:53.940784    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"
Jul 22 15:43:05.559137 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:05.559052    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"]
Jul 22 15:43:05.563340 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:05.563315    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e59bf8ff-57slg"]
Jul 22 15:43:07.020900 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:07.020865    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="605a8ceb-a25e-4a9a-8235-fd10d848d57e" path="/var/lib/kubelet/pods/605a8ceb-a25e-4a9a-8235-fd10d848d57e/volumes"
Jul 22 15:43:34.396850 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.396814    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"]
Jul 22 15:43:34.397338 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.397218    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605a8ceb-a25e-4a9a-8235-fd10d848d57e" containerName="revoke-keys"
Jul 22 15:43:34.397338 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.397228    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="605a8ceb-a25e-4a9a-8235-fd10d848d57e" containerName="revoke-keys"
Jul 22 15:43:34.397338 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.397326    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="605a8ceb-a25e-4a9a-8235-fd10d848d57e" containerName="revoke-keys"
Jul 22 15:43:34.400424 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.400406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:34.403004 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.402975    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:43:34.403166 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.402975    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:43:34.407335 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.407306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"]
Jul 22 15:43:34.440102 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.440063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ea89d6-c295-4d93-ac78-cf80c24b9498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k\" (UID: \"f1ea89d6-c295-4d93-ac78-cf80c24b9498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:34.540893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.540850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ea89d6-c295-4d93-ac78-cf80c24b9498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k\" (UID: \"f1ea89d6-c295-4d93-ac78-cf80c24b9498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:34.541570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.541550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ea89d6-c295-4d93-ac78-cf80c24b9498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k\" (UID: \"f1ea89d6-c295-4d93-ac78-cf80c24b9498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:34.713951 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.713859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:34.879122 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:34.879086    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"]
Jul 22 15:43:34.883188 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:43:34.883159    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ea89d6_c295_4d93_ac78_cf80c24b9498.slice/crio-31dff47d79fb6f34fbc6e2a31571aa4c2c4cc4cf2ec50c0da45c9c2e4a820636 WatchSource:0}: Error finding container 31dff47d79fb6f34fbc6e2a31571aa4c2c4cc4cf2ec50c0da45c9c2e4a820636: Status 404 returned error can't find the container with id 31dff47d79fb6f34fbc6e2a31571aa4c2c4cc4cf2ec50c0da45c9c2e4a820636
Jul 22 15:43:35.152083 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:35.152046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k" event={"ID":"f1ea89d6-c295-4d93-ac78-cf80c24b9498","Type":"ContainerStarted","Data":"31dff47d79fb6f34fbc6e2a31571aa4c2c4cc4cf2ec50c0da45c9c2e4a820636"}
Jul 22 15:43:36.158406 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:36.158368    2580 generic.go:356] "Generic (PLEG): container finished" podID="f1ea89d6-c295-4d93-ac78-cf80c24b9498" containerID="6fb4c3ba05f4f1165cddf8b4f56eeb19de7431f69cf64f3104b4694b66929018" exitCode=0
Jul 22 15:43:36.158843 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:36.158452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k" event={"ID":"f1ea89d6-c295-4d93-ac78-cf80c24b9498","Type":"ContainerDied","Data":"6fb4c3ba05f4f1165cddf8b4f56eeb19de7431f69cf64f3104b4694b66929018"}
Jul 22 15:43:37.305208 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:37.305178    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:37.367610 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:37.367572    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ea89d6-c295-4d93-ac78-cf80c24b9498-maas-api-service-ca\") pod \"f1ea89d6-c295-4d93-ac78-cf80c24b9498\" (UID: \"f1ea89d6-c295-4d93-ac78-cf80c24b9498\") "
Jul 22 15:43:37.367950 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:37.367925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ea89d6-c295-4d93-ac78-cf80c24b9498-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1ea89d6-c295-4d93-ac78-cf80c24b9498" (UID: "f1ea89d6-c295-4d93-ac78-cf80c24b9498"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:43:37.468866 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:37.468768    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ea89d6-c295-4d93-ac78-cf80c24b9498-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:43:38.170543 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:38.170483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k" event={"ID":"f1ea89d6-c295-4d93-ac78-cf80c24b9498","Type":"ContainerDied","Data":"31dff47d79fb6f34fbc6e2a31571aa4c2c4cc4cf2ec50c0da45c9c2e4a820636"}
Jul 22 15:43:38.170543 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:38.170527    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"
Jul 22 15:43:38.170779 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:38.170557    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31dff47d79fb6f34fbc6e2a31571aa4c2c4cc4cf2ec50c0da45c9c2e4a820636"
Jul 22 15:43:49.478105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:49.478069    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"]
Jul 22 15:43:49.484911 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:49.484872    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e6514ea7-2rg6k"]
Jul 22 15:43:51.028706 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:43:51.028666    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ea89d6-c295-4d93-ac78-cf80c24b9498" path="/var/lib/kubelet/pods/f1ea89d6-c295-4d93-ac78-cf80c24b9498/volumes"
Jul 22 15:44:18.691039 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.690997    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"]
Jul 22 15:44:18.691537 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.691459    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ea89d6-c295-4d93-ac78-cf80c24b9498" containerName="revoke-keys"
Jul 22 15:44:18.691537 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.691470    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ea89d6-c295-4d93-ac78-cf80c24b9498" containerName="revoke-keys"
Jul 22 15:44:18.691616 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.691586    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ea89d6-c295-4d93-ac78-cf80c24b9498" containerName="revoke-keys"
Jul 22 15:44:18.694708 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.694690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:18.699991 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.699968    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:44:18.699991 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.699970    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:44:18.704547 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.704522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"]
Jul 22 15:44:18.786370 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.786329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb\" (UID: \"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:18.887966 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.887922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb\" (UID: \"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:18.888584 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:18.888562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb\" (UID: \"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:19.006111 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:19.006072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:19.150396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:19.150364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"]
Jul 22 15:44:19.151872 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:44:19.151842    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047325ad_3ddc_4f8f_8aa9_c95f1c10f3ac.slice/crio-a2f37b50362023b7ec386274deb5c2bf41bde5a0851ee3650e8067f4c28b8ddd WatchSource:0}: Error finding container a2f37b50362023b7ec386274deb5c2bf41bde5a0851ee3650e8067f4c28b8ddd: Status 404 returned error can't find the container with id a2f37b50362023b7ec386274deb5c2bf41bde5a0851ee3650e8067f4c28b8ddd
Jul 22 15:44:19.379524 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:19.379470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb" event={"ID":"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac","Type":"ContainerStarted","Data":"a2f37b50362023b7ec386274deb5c2bf41bde5a0851ee3650e8067f4c28b8ddd"}
Jul 22 15:44:20.386508 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:20.386450    2580 generic.go:356] "Generic (PLEG): container finished" podID="047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac" containerID="4a3a1cb270839282cb57dc8f8dc945897a7c3df4ca2067d949d975afdb2c172e" exitCode=0
Jul 22 15:44:20.387023 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:20.386539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb" event={"ID":"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac","Type":"ContainerDied","Data":"4a3a1cb270839282cb57dc8f8dc945897a7c3df4ca2067d949d975afdb2c172e"}
Jul 22 15:44:21.530861 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:21.530836    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:21.613366 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:21.613326    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac-maas-api-service-ca\") pod \"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac\" (UID: \"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac\") "
Jul 22 15:44:21.613776 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:21.613755    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac" (UID: "047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:44:21.714198 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:21.714105    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:44:22.397582 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:22.397545    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"
Jul 22 15:44:22.397582 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:22.397562    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb" event={"ID":"047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac","Type":"ContainerDied","Data":"a2f37b50362023b7ec386274deb5c2bf41bde5a0851ee3650e8067f4c28b8ddd"}
Jul 22 15:44:22.397804 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:22.397616    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2f37b50362023b7ec386274deb5c2bf41bde5a0851ee3650e8067f4c28b8ddd"
Jul 22 15:44:33.797650 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:33.797560    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"]
Jul 22 15:44:33.799297 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:33.799274    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-37d7a1f9-qb2fb"]
Jul 22 15:44:35.021788 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:44:35.021756    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac" path="/var/lib/kubelet/pods/047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac/volumes"
Jul 22 15:45:00.166266 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.166232    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"]
Jul 22 15:45:00.167000 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.166979    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac" containerName="revoke-keys"
Jul 22 15:45:00.167000 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.167002    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac" containerName="revoke-keys"
Jul 22 15:45:00.167144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.167114    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="047325ad-3ddc-4f8f-8aa9-c95f1c10f3ac" containerName="revoke-keys"
Jul 22 15:45:00.170305 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.170284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"
Jul 22 15:45:00.174792 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.174766    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:45:00.182381 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.182361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"
Jul 22 15:45:00.238435 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.238401    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"]
Jul 22 15:45:00.524365 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.524338    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"]
Jul 22 15:45:00.526913 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:45:00.526886    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12614d2e_a2b6_4ba9_8694_26db49afde27.slice/crio-6b9bbfc61d2eb14781082141b0c1d4ebe2d587d28b73af2f4f28bb67d6a12038 WatchSource:0}: Error finding container 6b9bbfc61d2eb14781082141b0c1d4ebe2d587d28b73af2f4f28bb67d6a12038: Status 404 returned error can't find the container with id 6b9bbfc61d2eb14781082141b0c1d4ebe2d587d28b73af2f4f28bb67d6a12038
Jul 22 15:45:00.596123 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:00.596084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52" event={"ID":"12614d2e-a2b6-4ba9-8694-26db49afde27","Type":"ContainerStarted","Data":"6b9bbfc61d2eb14781082141b0c1d4ebe2d587d28b73af2f4f28bb67d6a12038"}
Jul 22 15:45:01.602834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:01.602799    2580 generic.go:356] "Generic (PLEG): container finished" podID="12614d2e-a2b6-4ba9-8694-26db49afde27" containerID="ff2ee3120e2334ee6961529f750f45b87fa687e0ed41b2ca3da284b48682b4ad" exitCode=0
Jul 22 15:45:01.603242 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:01.602871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52" event={"ID":"12614d2e-a2b6-4ba9-8694-26db49afde27","Type":"ContainerDied","Data":"ff2ee3120e2334ee6961529f750f45b87fa687e0ed41b2ca3da284b48682b4ad"}
Jul 22 15:45:02.745132 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:02.745106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"
Jul 22 15:45:03.614775 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:03.614746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52"
Jul 22 15:45:03.614962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:03.614793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-8pq52" event={"ID":"12614d2e-a2b6-4ba9-8694-26db49afde27","Type":"ContainerDied","Data":"6b9bbfc61d2eb14781082141b0c1d4ebe2d587d28b73af2f4f28bb67d6a12038"}
Jul 22 15:45:03.614962 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:45:03.614839    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b9bbfc61d2eb14781082141b0c1d4ebe2d587d28b73af2f4f28bb67d6a12038"
Jul 22 15:46:26.940238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.940197    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"]
Jul 22 15:46:26.940749 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.940653    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12614d2e-a2b6-4ba9-8694-26db49afde27" containerName="cleanup"
Jul 22 15:46:26.940749 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.940665    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="12614d2e-a2b6-4ba9-8694-26db49afde27" containerName="cleanup"
Jul 22 15:46:26.940749 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.940750    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="12614d2e-a2b6-4ba9-8694-26db49afde27" containerName="cleanup"
Jul 22 15:46:26.944011 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.943990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:26.946732 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.946708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:46:26.946886 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.946742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:46:26.952186 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:26.952155    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"]
Jul 22 15:46:27.097004 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:27.096964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3401686-845f-4ef0-be9e-a626e4060fa4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl\" (UID: \"c3401686-845f-4ef0-be9e-a626e4060fa4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:27.198174 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:27.198081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3401686-845f-4ef0-be9e-a626e4060fa4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl\" (UID: \"c3401686-845f-4ef0-be9e-a626e4060fa4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:27.198756 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:27.198736    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3401686-845f-4ef0-be9e-a626e4060fa4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl\" (UID: \"c3401686-845f-4ef0-be9e-a626e4060fa4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:27.257862 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:27.257833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:27.394372 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:27.394348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"]
Jul 22 15:46:27.396869 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:46:27.396838    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3401686_845f_4ef0_be9e_a626e4060fa4.slice/crio-58d20307bb63ddbb09136385fc1bf06dfc5d9a94568247493b0258e3adac6c5f WatchSource:0}: Error finding container 58d20307bb63ddbb09136385fc1bf06dfc5d9a94568247493b0258e3adac6c5f: Status 404 returned error can't find the container with id 58d20307bb63ddbb09136385fc1bf06dfc5d9a94568247493b0258e3adac6c5f
Jul 22 15:46:28.056199 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:28.056161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl" event={"ID":"c3401686-845f-4ef0-be9e-a626e4060fa4","Type":"ContainerStarted","Data":"65ccd39c1509c52ebacb843268568bd2ea4db9b08d30715ff3f9149a0f64cdca"}
Jul 22 15:46:28.056622 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:28.056222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl" event={"ID":"c3401686-845f-4ef0-be9e-a626e4060fa4","Type":"ContainerStarted","Data":"58d20307bb63ddbb09136385fc1bf06dfc5d9a94568247493b0258e3adac6c5f"}
Jul 22 15:46:29.062728 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:29.062696    2580 generic.go:356] "Generic (PLEG): container finished" podID="c3401686-845f-4ef0-be9e-a626e4060fa4" containerID="65ccd39c1509c52ebacb843268568bd2ea4db9b08d30715ff3f9149a0f64cdca" exitCode=0
Jul 22 15:46:29.063141 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:29.062763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl" event={"ID":"c3401686-845f-4ef0-be9e-a626e4060fa4","Type":"ContainerDied","Data":"65ccd39c1509c52ebacb843268568bd2ea4db9b08d30715ff3f9149a0f64cdca"}
Jul 22 15:46:30.210044 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:30.210019    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:30.331733 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:30.331633    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3401686-845f-4ef0-be9e-a626e4060fa4-maas-api-service-ca\") pod \"c3401686-845f-4ef0-be9e-a626e4060fa4\" (UID: \"c3401686-845f-4ef0-be9e-a626e4060fa4\") "
Jul 22 15:46:30.331963 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:30.331941    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3401686-845f-4ef0-be9e-a626e4060fa4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3401686-845f-4ef0-be9e-a626e4060fa4" (UID: "c3401686-845f-4ef0-be9e-a626e4060fa4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:46:30.432955 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:30.432921    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3401686-845f-4ef0-be9e-a626e4060fa4-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:46:31.074973 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:31.074928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl" event={"ID":"c3401686-845f-4ef0-be9e-a626e4060fa4","Type":"ContainerDied","Data":"58d20307bb63ddbb09136385fc1bf06dfc5d9a94568247493b0258e3adac6c5f"}
Jul 22 15:46:31.074973 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:31.074956    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"
Jul 22 15:46:31.075177 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:31.075026    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58d20307bb63ddbb09136385fc1bf06dfc5d9a94568247493b0258e3adac6c5f"
Jul 22 15:46:42.013367 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:42.013327    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"]
Jul 22 15:46:42.016579 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:42.016551    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97d19d49-4zkcl"]
Jul 22 15:46:43.022253 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:46:43.022216    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3401686-845f-4ef0-be9e-a626e4060fa4" path="/var/lib/kubelet/pods/c3401686-845f-4ef0-be9e-a626e4060fa4/volumes"
Jul 22 15:47:23.817852 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.817792    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"]
Jul 22 15:47:23.820410 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.818380    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3401686-845f-4ef0-be9e-a626e4060fa4" containerName="revoke-keys"
Jul 22 15:47:23.820410 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.818398    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3401686-845f-4ef0-be9e-a626e4060fa4" containerName="revoke-keys"
Jul 22 15:47:23.820410 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.818479    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3401686-845f-4ef0-be9e-a626e4060fa4" containerName="revoke-keys"
Jul 22 15:47:23.821606 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.821585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:23.825374 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.825347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b432cbd3\"/\"kube-root-ca.crt\""
Jul 22 15:47:23.825374 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.825363    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b432cbd3\"/\"e2e-lifecycle-model-b432cbd3-kserve-self-signed-certs\""
Jul 22 15:47:23.825570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.825381    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b432cbd3\"/\"default-dockercfg-zhzq5\""
Jul 22 15:47:23.825570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.825392    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b432cbd3\"/\"openshift-service-ca.crt\""
Jul 22 15:47:23.831035 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.831017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"]
Jul 22 15:47:23.936108 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.936070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jddc\" (UniqueName: \"kubernetes.io/projected/ddcce8be-1326-48a8-b3c1-0c5b15167a55-kube-api-access-4jddc\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:23.936108 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.936113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tmp-dir\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:23.936327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.936166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tls-certs\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:23.936327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.936227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-home\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:23.936327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.936254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-model-cache\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:23.936327 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:23.936290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-dshm\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037083 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jddc\" (UniqueName: \"kubernetes.io/projected/ddcce8be-1326-48a8-b3c1-0c5b15167a55-kube-api-access-4jddc\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037083 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tmp-dir\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037347 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tls-certs\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037347 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-home\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037347 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-model-cache\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037347 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-dshm\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037608 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tmp-dir\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037716 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-model-cache\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.037784 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.037700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-home\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.039775 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.039755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-dshm\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.040018 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.040002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tls-certs\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.045705 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.045686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jddc\" (UniqueName: \"kubernetes.io/projected/ddcce8be-1326-48a8-b3c1-0c5b15167a55-kube-api-access-4jddc\") pod \"e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") " pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.132037 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.131931    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:24.273396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.273371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"]
Jul 22 15:47:24.275376 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:47:24.275346    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddcce8be_1326_48a8_b3c1_0c5b15167a55.slice/crio-7614e3063501c2e9bb102b253879424c7c06ef905cf4c095b123d545f22c4252 WatchSource:0}: Error finding container 7614e3063501c2e9bb102b253879424c7c06ef905cf4c095b123d545f22c4252: Status 404 returned error can't find the container with id 7614e3063501c2e9bb102b253879424c7c06ef905cf4c095b123d545f22c4252
Jul 22 15:47:24.369715 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.369673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg" event={"ID":"ddcce8be-1326-48a8-b3c1-0c5b15167a55","Type":"ContainerStarted","Data":"c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850"}
Jul 22 15:47:24.369715 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.369723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg" event={"ID":"ddcce8be-1326-48a8-b3c1-0c5b15167a55","Type":"ContainerStarted","Data":"7614e3063501c2e9bb102b253879424c7c06ef905cf4c095b123d545f22c4252"}
Jul 22 15:47:24.388776 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:24.388661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg" podStartSLOduration=1.388645436 podStartE2EDuration="1.388645436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:47:24.386437685 +0000 UTC m=+2053.828521132" watchObservedRunningTime="2026-07-22 15:47:24.388645436 +0000 UTC m=+2053.830728883"
Jul 22 15:47:34.133237 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:34.133147    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:34.133237 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:34.133205    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:34.145859 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:34.145827    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:34.435791 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:34.435704    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:47:49.314993 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.314955    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"]
Jul 22 15:47:49.318730 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.318709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:49.321196 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.321174    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:47:49.321351 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.321286    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:47:49.326589 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.326563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"]
Jul 22 15:47:49.378182 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.378140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6587aa39-aea6-4dbc-afcd-94993b6d2a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w\" (UID: \"6587aa39-aea6-4dbc-afcd-94993b6d2a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:49.479190 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.479150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6587aa39-aea6-4dbc-afcd-94993b6d2a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w\" (UID: \"6587aa39-aea6-4dbc-afcd-94993b6d2a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:49.479822 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.479803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6587aa39-aea6-4dbc-afcd-94993b6d2a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w\" (UID: \"6587aa39-aea6-4dbc-afcd-94993b6d2a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:49.632541 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.632420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:49.779937 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:49.779913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"]
Jul 22 15:47:49.782539 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:47:49.782506    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6587aa39_aea6_4dbc_afcd_94993b6d2a55.slice/crio-1a0a59375a9c0f81dee0688d9443a8374dc03d054b066cfa45a919be07aebf57 WatchSource:0}: Error finding container 1a0a59375a9c0f81dee0688d9443a8374dc03d054b066cfa45a919be07aebf57: Status 404 returned error can't find the container with id 1a0a59375a9c0f81dee0688d9443a8374dc03d054b066cfa45a919be07aebf57
Jul 22 15:47:50.516773 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:50.516743    2580 generic.go:356] "Generic (PLEG): container finished" podID="6587aa39-aea6-4dbc-afcd-94993b6d2a55" containerID="6f4b374dcf92f9a720b0e64a6ee7db68702db6df6da01bc7f3b629b45dc9d9b1" exitCode=0
Jul 22 15:47:50.517130 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:50.516821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w" event={"ID":"6587aa39-aea6-4dbc-afcd-94993b6d2a55","Type":"ContainerDied","Data":"6f4b374dcf92f9a720b0e64a6ee7db68702db6df6da01bc7f3b629b45dc9d9b1"}
Jul 22 15:47:50.517130 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:50.516873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w" event={"ID":"6587aa39-aea6-4dbc-afcd-94993b6d2a55","Type":"ContainerStarted","Data":"1a0a59375a9c0f81dee0688d9443a8374dc03d054b066cfa45a919be07aebf57"}
Jul 22 15:47:51.664827 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:51.664793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:51.802788 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:51.802695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6587aa39-aea6-4dbc-afcd-94993b6d2a55-maas-api-service-ca\") pod \"6587aa39-aea6-4dbc-afcd-94993b6d2a55\" (UID: \"6587aa39-aea6-4dbc-afcd-94993b6d2a55\") "
Jul 22 15:47:51.803043 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:51.803021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6587aa39-aea6-4dbc-afcd-94993b6d2a55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6587aa39-aea6-4dbc-afcd-94993b6d2a55" (UID: "6587aa39-aea6-4dbc-afcd-94993b6d2a55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:47:51.904142 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:51.904101    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6587aa39-aea6-4dbc-afcd-94993b6d2a55-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:47:52.529109 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:52.529074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w" event={"ID":"6587aa39-aea6-4dbc-afcd-94993b6d2a55","Type":"ContainerDied","Data":"1a0a59375a9c0f81dee0688d9443a8374dc03d054b066cfa45a919be07aebf57"}
Jul 22 15:47:52.529109 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:52.529098    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"
Jul 22 15:47:52.529328 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:47:52.529123    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a0a59375a9c0f81dee0688d9443a8374dc03d054b066cfa45a919be07aebf57"
Jul 22 15:48:09.404005 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:09.403960    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"]
Jul 22 15:48:09.406444 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:09.406416    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b432cbd3-ls52w"]
Jul 22 15:48:11.026136 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:11.026095    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6587aa39-aea6-4dbc-afcd-94993b6d2a55" path="/var/lib/kubelet/pods/6587aa39-aea6-4dbc-afcd-94993b6d2a55/volumes"
Jul 22 15:48:11.171272 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:11.171237    2580 scope.go:117] "RemoveContainer" containerID="e06418e740daadcb4f767aabf712e582ff144de4be851977e46c94ca3ebba57d"
Jul 22 15:48:16.685144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:16.685096    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"]
Jul 22 15:48:31.708968 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:31.708925    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg" podUID="ddcce8be-1326-48a8-b3c1-0c5b15167a55" containerName="main" containerID="cri-o://c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850" gracePeriod=45
Jul 22 15:48:31.960976 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:31.960907    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:48:32.097756 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.097713    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-dshm\") pod \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") "
Jul 22 15:48:32.097938 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.097800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-home\") pod \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") "
Jul 22 15:48:32.097938 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.097829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tls-certs\") pod \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") "
Jul 22 15:48:32.097938 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.097853    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-model-cache\") pod \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") "
Jul 22 15:48:32.097938 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.097883    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tmp-dir\") pod \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") "
Jul 22 15:48:32.098172 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.097983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jddc\" (UniqueName: \"kubernetes.io/projected/ddcce8be-1326-48a8-b3c1-0c5b15167a55-kube-api-access-4jddc\") pod \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\" (UID: \"ddcce8be-1326-48a8-b3c1-0c5b15167a55\") "
Jul 22 15:48:32.098172 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.098109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-home" (OuterVolumeSpecName: "home") pod "ddcce8be-1326-48a8-b3c1-0c5b15167a55" (UID: "ddcce8be-1326-48a8-b3c1-0c5b15167a55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:32.098172 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.098125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-model-cache" (OuterVolumeSpecName: "model-cache") pod "ddcce8be-1326-48a8-b3c1-0c5b15167a55" (UID: "ddcce8be-1326-48a8-b3c1-0c5b15167a55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:32.098337 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.098170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ddcce8be-1326-48a8-b3c1-0c5b15167a55" (UID: "ddcce8be-1326-48a8-b3c1-0c5b15167a55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:32.098403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.098360    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tmp-dir\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:48:32.098403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.098374    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-home\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:48:32.098403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.098383    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-model-cache\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:48:32.100319 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.100297    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-dshm" (OuterVolumeSpecName: "dshm") pod "ddcce8be-1326-48a8-b3c1-0c5b15167a55" (UID: "ddcce8be-1326-48a8-b3c1-0c5b15167a55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:32.100415 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.100332    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ddcce8be-1326-48a8-b3c1-0c5b15167a55" (UID: "ddcce8be-1326-48a8-b3c1-0c5b15167a55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:48:32.100454 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.100413    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddcce8be-1326-48a8-b3c1-0c5b15167a55-kube-api-access-4jddc" (OuterVolumeSpecName: "kube-api-access-4jddc") pod "ddcce8be-1326-48a8-b3c1-0c5b15167a55" (UID: "ddcce8be-1326-48a8-b3c1-0c5b15167a55"). InnerVolumeSpecName "kube-api-access-4jddc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:32.199471 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.199416    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jddc\" (UniqueName: \"kubernetes.io/projected/ddcce8be-1326-48a8-b3c1-0c5b15167a55-kube-api-access-4jddc\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:48:32.199471 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.199465    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddcce8be-1326-48a8-b3c1-0c5b15167a55-dshm\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:48:32.199471 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.199479    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddcce8be-1326-48a8-b3c1-0c5b15167a55-tls-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:48:32.752594 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.752554    2580 generic.go:356] "Generic (PLEG): container finished" podID="ddcce8be-1326-48a8-b3c1-0c5b15167a55" containerID="c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850" exitCode=0
Jul 22 15:48:32.753017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.752606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"
Jul 22 15:48:32.753017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.752645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg" event={"ID":"ddcce8be-1326-48a8-b3c1-0c5b15167a55","Type":"ContainerDied","Data":"c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850"}
Jul 22 15:48:32.753017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.752691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg" event={"ID":"ddcce8be-1326-48a8-b3c1-0c5b15167a55","Type":"ContainerDied","Data":"7614e3063501c2e9bb102b253879424c7c06ef905cf4c095b123d545f22c4252"}
Jul 22 15:48:32.753017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.752709    2580 scope.go:117] "RemoveContainer" containerID="c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850"
Jul 22 15:48:32.762392 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.762366    2580 scope.go:117] "RemoveContainer" containerID="c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850"
Jul 22 15:48:32.762736 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:48:32.762716    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850\": container with ID starting with c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850 not found: ID does not exist" containerID="c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850"
Jul 22 15:48:32.762823 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.762753    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850"} err="failed to get container status \"c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850\": rpc error: code = NotFound desc = could not find container \"c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850\": container with ID starting with c576b2e5ac97cea52a1477fdc1633b906a9694ed9ea8abccd25b66d80d74b850 not found: ID does not exist"
Jul 22 15:48:32.776070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.776036    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"]
Jul 22 15:48:32.780877 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:32.780846    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b432cbd3/e2e-lifecycle-model-b432cbd3-kserve-f4cdcc587-6r8tg"]
Jul 22 15:48:33.021982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:48:33.021904    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddcce8be-1326-48a8-b3c1-0c5b15167a55" path="/var/lib/kubelet/pods/ddcce8be-1326-48a8-b3c1-0c5b15167a55/volumes"
Jul 22 15:49:11.194045 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:49:11.193801    2580 scope.go:117] "RemoveContainer" containerID="bb5c4c91dd2ab65a60d05b9d92e182ff1829f324b3c20415f45d9ad7edcf60e3"
Jul 22 15:50:11.218264 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:11.218230    2580 scope.go:117] "RemoveContainer" containerID="6fb4c3ba05f4f1165cddf8b4f56eeb19de7431f69cf64f3104b4694b66929018"
Jul 22 15:50:28.323124 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.323086    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"]
Jul 22 15:50:28.323790 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.323771    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6587aa39-aea6-4dbc-afcd-94993b6d2a55" containerName="revoke-keys"
Jul 22 15:50:28.323873 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.323794    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6587aa39-aea6-4dbc-afcd-94993b6d2a55" containerName="revoke-keys"
Jul 22 15:50:28.323873 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.323835    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddcce8be-1326-48a8-b3c1-0c5b15167a55" containerName="main"
Jul 22 15:50:28.323873 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.323844    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddcce8be-1326-48a8-b3c1-0c5b15167a55" containerName="main"
Jul 22 15:50:28.324027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.323985    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6587aa39-aea6-4dbc-afcd-94993b6d2a55" containerName="revoke-keys"
Jul 22 15:50:28.324027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.324001    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddcce8be-1326-48a8-b3c1-0c5b15167a55" containerName="main"
Jul 22 15:50:28.327430 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.327410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.331124 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.331090    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c24af7\"/\"test-model-c24af7-kserve-self-signed-certs\""
Jul 22 15:50:28.331246 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.331090    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c24af7\"/\"kube-root-ca.crt\""
Jul 22 15:50:28.331246 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.331090    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c24af7\"/\"openshift-service-ca.crt\""
Jul 22 15:50:28.331246 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.331090    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c24af7\"/\"default-dockercfg-5hts6\""
Jul 22 15:50:28.337228 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.337202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"]
Jul 22 15:50:28.443573 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.443536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/5ef6bca0-04ad-4610-9197-44cad131ef86-kube-api-access-r4gwz\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.443793 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.443601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-tmp-dir\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.443793 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.443651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6bca0-04ad-4610-9197-44cad131ef86-tls-certs\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.443793 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.443731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-home\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.443793 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.443757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-model-cache\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.443974 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.443816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-dshm\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545060 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/5ef6bca0-04ad-4610-9197-44cad131ef86-kube-api-access-r4gwz\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545272 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-tmp-dir\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545272 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6bca0-04ad-4610-9197-44cad131ef86-tls-certs\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545272 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-home\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545445 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-model-cache\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-dshm\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545598 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-tmp-dir\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545653 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-home\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.545746 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.545728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-model-cache\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.547837 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.547815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-dshm\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.548017 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.547998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6bca0-04ad-4610-9197-44cad131ef86-tls-certs\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.554063 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.554037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/5ef6bca0-04ad-4610-9197-44cad131ef86-kube-api-access-r4gwz\") pod \"test-model-c24af7-kserve-5b944888f9-6lv8t\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") " pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.637604 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.637498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:28.782883 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:28.782858    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"]
Jul 22 15:50:28.785318 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:50:28.785286    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ef6bca0_04ad_4610_9197_44cad131ef86.slice/crio-be42f4d8f4a7a32edb6c6113da2a70641754a4950b51a4586f665376de32a62e WatchSource:0}: Error finding container be42f4d8f4a7a32edb6c6113da2a70641754a4950b51a4586f665376de32a62e: Status 404 returned error can't find the container with id be42f4d8f4a7a32edb6c6113da2a70641754a4950b51a4586f665376de32a62e
Jul 22 15:50:29.371247 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:29.371204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t" event={"ID":"5ef6bca0-04ad-4610-9197-44cad131ef86","Type":"ContainerStarted","Data":"63f22690dfac606fb0ec4a738067b69b1e3994128408a1801cbf2463289837d1"}
Jul 22 15:50:29.371247 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:29.371254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t" event={"ID":"5ef6bca0-04ad-4610-9197-44cad131ef86","Type":"ContainerStarted","Data":"be42f4d8f4a7a32edb6c6113da2a70641754a4950b51a4586f665376de32a62e"}
Jul 22 15:50:29.394980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:29.394923    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t" podStartSLOduration=1.394908191 podStartE2EDuration="1.394908191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:50:29.392455729 +0000 UTC m=+2238.834539176" watchObservedRunningTime="2026-07-22 15:50:29.394908191 +0000 UTC m=+2238.836991638"
Jul 22 15:50:38.638090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:38.637991    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:38.638090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:38.638069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:38.650674 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:38.650646    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:50:39.432070 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:50:39.432039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:51:11.239096 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:11.239055    2580 scope.go:117] "RemoveContainer" containerID="4a3a1cb270839282cb57dc8f8dc945897a7c3df4ca2067d949d975afdb2c172e"
Jul 22 15:51:31.551268 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:31.551227    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"]
Jul 22 15:51:32.624510 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.624452    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"]
Jul 22 15:51:32.628451 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.628432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:32.631221 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.631196    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:51:32.631376 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.631196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:51:32.637373 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.637341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"]
Jul 22 15:51:32.758625 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.758573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f85edf1-3ea7-4b61-b2a7-74d425452740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5\" (UID: \"7f85edf1-3ea7-4b61-b2a7-74d425452740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:32.860064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.860024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f85edf1-3ea7-4b61-b2a7-74d425452740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5\" (UID: \"7f85edf1-3ea7-4b61-b2a7-74d425452740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:32.860790 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.860768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f85edf1-3ea7-4b61-b2a7-74d425452740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5\" (UID: \"7f85edf1-3ea7-4b61-b2a7-74d425452740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:32.939876 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:32.939774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:33.086784 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:33.086757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"]
Jul 22 15:51:33.088867 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:51:33.088833    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f85edf1_3ea7_4b61_b2a7_74d425452740.slice/crio-5b55a8ca0980f8a3b516330735cf191608badf5655a3df6ff6ad78a4a612a69a WatchSource:0}: Error finding container 5b55a8ca0980f8a3b516330735cf191608badf5655a3df6ff6ad78a4a612a69a: Status 404 returned error can't find the container with id 5b55a8ca0980f8a3b516330735cf191608badf5655a3df6ff6ad78a4a612a69a
Jul 22 15:51:33.700709 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:33.700615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5" event={"ID":"7f85edf1-3ea7-4b61-b2a7-74d425452740","Type":"ContainerStarted","Data":"3ec0ae196791caa7a8315afd1ea6b59eeb91d9854acfc949bd3c34f465f90c1d"}
Jul 22 15:51:33.700709 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:33.700671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5" event={"ID":"7f85edf1-3ea7-4b61-b2a7-74d425452740","Type":"ContainerStarted","Data":"5b55a8ca0980f8a3b516330735cf191608badf5655a3df6ff6ad78a4a612a69a"}
Jul 22 15:51:33.718695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:33.718636    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5" podStartSLOduration=1.7186183339999999 podStartE2EDuration="1.718618334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:51:33.717368637 +0000 UTC m=+2303.159452085" watchObservedRunningTime="2026-07-22 15:51:33.718618334 +0000 UTC m=+2303.160701782"
Jul 22 15:51:34.708115 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:34.708081    2580 generic.go:356] "Generic (PLEG): container finished" podID="7f85edf1-3ea7-4b61-b2a7-74d425452740" containerID="3ec0ae196791caa7a8315afd1ea6b59eeb91d9854acfc949bd3c34f465f90c1d" exitCode=0
Jul 22 15:51:34.708543 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:34.708165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5" event={"ID":"7f85edf1-3ea7-4b61-b2a7-74d425452740","Type":"ContainerDied","Data":"3ec0ae196791caa7a8315afd1ea6b59eeb91d9854acfc949bd3c34f465f90c1d"}
Jul 22 15:51:35.862674 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:35.862635    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:35.992219 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:35.992121    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f85edf1-3ea7-4b61-b2a7-74d425452740-maas-api-service-ca\") pod \"7f85edf1-3ea7-4b61-b2a7-74d425452740\" (UID: \"7f85edf1-3ea7-4b61-b2a7-74d425452740\") "
Jul 22 15:51:35.992551 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:35.992526    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f85edf1-3ea7-4b61-b2a7-74d425452740-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f85edf1-3ea7-4b61-b2a7-74d425452740" (UID: "7f85edf1-3ea7-4b61-b2a7-74d425452740"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:51:36.093750 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:36.093713    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f85edf1-3ea7-4b61-b2a7-74d425452740-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:36.720631 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:36.720603    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"
Jul 22 15:51:36.720804 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:36.720602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5" event={"ID":"7f85edf1-3ea7-4b61-b2a7-74d425452740","Type":"ContainerDied","Data":"5b55a8ca0980f8a3b516330735cf191608badf5655a3df6ff6ad78a4a612a69a"}
Jul 22 15:51:36.720804 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:36.720731    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b55a8ca0980f8a3b516330735cf191608badf5655a3df6ff6ad78a4a612a69a"
Jul 22 15:51:46.572461 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:46.572406    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t" podUID="5ef6bca0-04ad-4610-9197-44cad131ef86" containerName="main" containerID="cri-o://63f22690dfac606fb0ec4a738067b69b1e3994128408a1801cbf2463289837d1" gracePeriod=45
Jul 22 15:51:46.775977 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:46.775945    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ef6bca0-04ad-4610-9197-44cad131ef86" containerID="63f22690dfac606fb0ec4a738067b69b1e3994128408a1801cbf2463289837d1" exitCode=0
Jul 22 15:51:46.776127 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:46.776016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t" event={"ID":"5ef6bca0-04ad-4610-9197-44cad131ef86","Type":"ContainerDied","Data":"63f22690dfac606fb0ec4a738067b69b1e3994128408a1801cbf2463289837d1"}
Jul 22 15:51:46.838332 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:46.838264    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:51:47.006624 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006591    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/5ef6bca0-04ad-4610-9197-44cad131ef86-kube-api-access-r4gwz\") pod \"5ef6bca0-04ad-4610-9197-44cad131ef86\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") "
Jul 22 15:51:47.006834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6bca0-04ad-4610-9197-44cad131ef86-tls-certs\") pod \"5ef6bca0-04ad-4610-9197-44cad131ef86\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") "
Jul 22 15:51:47.006834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006702    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-model-cache\") pod \"5ef6bca0-04ad-4610-9197-44cad131ef86\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") "
Jul 22 15:51:47.006959 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-dshm\") pod \"5ef6bca0-04ad-4610-9197-44cad131ef86\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") "
Jul 22 15:51:47.006959 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-tmp-dir\") pod \"5ef6bca0-04ad-4610-9197-44cad131ef86\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") "
Jul 22 15:51:47.006959 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ef6bca0-04ad-4610-9197-44cad131ef86" (UID: "5ef6bca0-04ad-4610-9197-44cad131ef86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:51:47.006959 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.006940    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-home\") pod \"5ef6bca0-04ad-4610-9197-44cad131ef86\" (UID: \"5ef6bca0-04ad-4610-9197-44cad131ef86\") "
Jul 22 15:51:47.007152 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.007121    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-home" (OuterVolumeSpecName: "home") pod "5ef6bca0-04ad-4610-9197-44cad131ef86" (UID: "5ef6bca0-04ad-4610-9197-44cad131ef86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:51:47.007193 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.007159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ef6bca0-04ad-4610-9197-44cad131ef86" (UID: "5ef6bca0-04ad-4610-9197-44cad131ef86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:51:47.007585 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.007525    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-tmp-dir\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:47.007649 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.007589    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-home\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:47.007649 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.007604    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-model-cache\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:47.009200 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.009173    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef6bca0-04ad-4610-9197-44cad131ef86-kube-api-access-r4gwz" (OuterVolumeSpecName: "kube-api-access-r4gwz") pod "5ef6bca0-04ad-4610-9197-44cad131ef86" (UID: "5ef6bca0-04ad-4610-9197-44cad131ef86"). InnerVolumeSpecName "kube-api-access-r4gwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:51:47.009200 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.009193    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-dshm" (OuterVolumeSpecName: "dshm") pod "5ef6bca0-04ad-4610-9197-44cad131ef86" (UID: "5ef6bca0-04ad-4610-9197-44cad131ef86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:51:47.009350 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.009206    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ef6bca0-04ad-4610-9197-44cad131ef86-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ef6bca0-04ad-4610-9197-44cad131ef86" (UID: "5ef6bca0-04ad-4610-9197-44cad131ef86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:51:47.108980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.108877    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef6bca0-04ad-4610-9197-44cad131ef86-dshm\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:47.108980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.108921    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4gwz\" (UniqueName: \"kubernetes.io/projected/5ef6bca0-04ad-4610-9197-44cad131ef86-kube-api-access-r4gwz\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:47.108980 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.108937    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6bca0-04ad-4610-9197-44cad131ef86-tls-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:47.712983 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.712946    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"]
Jul 22 15:51:47.715453 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.715425    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c24af7-68fz5"]
Jul 22 15:51:47.782996 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.782958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t" event={"ID":"5ef6bca0-04ad-4610-9197-44cad131ef86","Type":"ContainerDied","Data":"be42f4d8f4a7a32edb6c6113da2a70641754a4950b51a4586f665376de32a62e"}
Jul 22 15:51:47.782996 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.782993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"
Jul 22 15:51:47.783243 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.783023    2580 scope.go:117] "RemoveContainer" containerID="63f22690dfac606fb0ec4a738067b69b1e3994128408a1801cbf2463289837d1"
Jul 22 15:51:47.799577 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.799548    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"]
Jul 22 15:51:47.806898 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:47.806872    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c24af7/test-model-c24af7-kserve-5b944888f9-6lv8t"]
Jul 22 15:51:49.021521 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:49.021455    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ef6bca0-04ad-4610-9197-44cad131ef86" path="/var/lib/kubelet/pods/5ef6bca0-04ad-4610-9197-44cad131ef86/volumes"
Jul 22 15:51:49.022022 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:49.021982    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f85edf1-3ea7-4b61-b2a7-74d425452740" path="/var/lib/kubelet/pods/7f85edf1-3ea7-4b61-b2a7-74d425452740/volumes"
Jul 22 15:51:54.525098 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525056    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"]
Jul 22 15:51:54.525595 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525580    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f85edf1-3ea7-4b61-b2a7-74d425452740" containerName="revoke-keys"
Jul 22 15:51:54.525655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525597    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f85edf1-3ea7-4b61-b2a7-74d425452740" containerName="revoke-keys"
Jul 22 15:51:54.525655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525608    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ef6bca0-04ad-4610-9197-44cad131ef86" containerName="main"
Jul 22 15:51:54.525655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525613    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ef6bca0-04ad-4610-9197-44cad131ef86" containerName="main"
Jul 22 15:51:54.525755 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525681    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ef6bca0-04ad-4610-9197-44cad131ef86" containerName="main"
Jul 22 15:51:54.525755 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.525690    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f85edf1-3ea7-4b61-b2a7-74d425452740" containerName="revoke-keys"
Jul 22 15:51:54.530128 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.530113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:54.532715 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.532668    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:51:54.532955 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.532744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:51:54.536196 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.536172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"]
Jul 22 15:51:54.680216 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.680175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e821fb4-be7f-4b54-859f-08f579041fc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5\" (UID: \"4e821fb4-be7f-4b54-859f-08f579041fc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:54.781311 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.781212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e821fb4-be7f-4b54-859f-08f579041fc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5\" (UID: \"4e821fb4-be7f-4b54-859f-08f579041fc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:54.781892 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.781872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e821fb4-be7f-4b54-859f-08f579041fc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5\" (UID: \"4e821fb4-be7f-4b54-859f-08f579041fc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:54.843272 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.843232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:54.987882 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:54.987847    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"]
Jul 22 15:51:54.988806 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:51:54.988775    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e821fb4_be7f_4b54_859f_08f579041fc1.slice/crio-f2f3c8dc1c7172d131881d91282162dbec4ad22d949d232541307e94166fb61b WatchSource:0}: Error finding container f2f3c8dc1c7172d131881d91282162dbec4ad22d949d232541307e94166fb61b: Status 404 returned error can't find the container with id f2f3c8dc1c7172d131881d91282162dbec4ad22d949d232541307e94166fb61b
Jul 22 15:51:55.831764 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:55.831726    2580 generic.go:356] "Generic (PLEG): container finished" podID="4e821fb4-be7f-4b54-859f-08f579041fc1" containerID="5900c58aae10b1a94c01572b0665618e216efc2026808767a7d2214628792d2a" exitCode=0
Jul 22 15:51:55.832166 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:55.831815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5" event={"ID":"4e821fb4-be7f-4b54-859f-08f579041fc1","Type":"ContainerDied","Data":"5900c58aae10b1a94c01572b0665618e216efc2026808767a7d2214628792d2a"}
Jul 22 15:51:55.832166 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:55.831867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5" event={"ID":"4e821fb4-be7f-4b54-859f-08f579041fc1","Type":"ContainerStarted","Data":"f2f3c8dc1c7172d131881d91282162dbec4ad22d949d232541307e94166fb61b"}
Jul 22 15:51:56.976695 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:56.976668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:57.003878 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:57.003849    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e821fb4-be7f-4b54-859f-08f579041fc1-maas-api-service-ca\") pod \"4e821fb4-be7f-4b54-859f-08f579041fc1\" (UID: \"4e821fb4-be7f-4b54-859f-08f579041fc1\") "
Jul 22 15:51:57.004297 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:57.004266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e821fb4-be7f-4b54-859f-08f579041fc1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e821fb4-be7f-4b54-859f-08f579041fc1" (UID: "4e821fb4-be7f-4b54-859f-08f579041fc1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:51:57.105581 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:57.105443    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e821fb4-be7f-4b54-859f-08f579041fc1-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:51:57.844105 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:57.844071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5" event={"ID":"4e821fb4-be7f-4b54-859f-08f579041fc1","Type":"ContainerDied","Data":"f2f3c8dc1c7172d131881d91282162dbec4ad22d949d232541307e94166fb61b"}
Jul 22 15:51:57.844290 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:57.844112    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"
Jul 22 15:51:57.844290 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:51:57.844120    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2f3c8dc1c7172d131881d91282162dbec4ad22d949d232541307e94166fb61b"
Jul 22 15:52:09.618245 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:09.618161    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"]
Jul 22 15:52:09.622418 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:09.622392    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cdfa08-4brv5"]
Jul 22 15:52:11.021840 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:11.021808    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e821fb4-be7f-4b54-859f-08f579041fc1" path="/var/lib/kubelet/pods/4e821fb4-be7f-4b54-859f-08f579041fc1/volumes"
Jul 22 15:52:40.417599 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.417557    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"]
Jul 22 15:52:40.420283 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.417983    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e821fb4-be7f-4b54-859f-08f579041fc1" containerName="revoke-keys"
Jul 22 15:52:40.420283 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.417994    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e821fb4-be7f-4b54-859f-08f579041fc1" containerName="revoke-keys"
Jul 22 15:52:40.420283 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.418072    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e821fb4-be7f-4b54-859f-08f579041fc1" containerName="revoke-keys"
Jul 22 15:52:40.421090 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.421073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:40.423554 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.423523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:52:40.423690 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.423592    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:52:40.429536 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.429474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"]
Jul 22 15:52:40.438737 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.438704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d77088-3b22-4472-a801-0aa478c3906b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-063391-b6765\" (UID: \"78d77088-3b22-4472-a801-0aa478c3906b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:40.539966 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.539932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d77088-3b22-4472-a801-0aa478c3906b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-063391-b6765\" (UID: \"78d77088-3b22-4472-a801-0aa478c3906b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:40.540608 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.540588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d77088-3b22-4472-a801-0aa478c3906b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-063391-b6765\" (UID: \"78d77088-3b22-4472-a801-0aa478c3906b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:40.733712 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.733616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:40.877128 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:40.877096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"]
Jul 22 15:52:40.879124 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:52:40.879095    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d77088_3b22_4472_a801_0aa478c3906b.slice/crio-9854ec12af323dce0dfe0bc198c5dfe950fa31d5da0a63463186ac978913cb87 WatchSource:0}: Error finding container 9854ec12af323dce0dfe0bc198c5dfe950fa31d5da0a63463186ac978913cb87: Status 404 returned error can't find the container with id 9854ec12af323dce0dfe0bc198c5dfe950fa31d5da0a63463186ac978913cb87
Jul 22 15:52:41.086163 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:41.086127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765" event={"ID":"78d77088-3b22-4472-a801-0aa478c3906b","Type":"ContainerStarted","Data":"9854ec12af323dce0dfe0bc198c5dfe950fa31d5da0a63463186ac978913cb87"}
Jul 22 15:52:42.093112 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:42.093072    2580 generic.go:356] "Generic (PLEG): container finished" podID="78d77088-3b22-4472-a801-0aa478c3906b" containerID="4e0f05c44e609fa2494749f9b3527937cc611991c7a49bbce76d4f22551e2f2b" exitCode=0
Jul 22 15:52:42.093576 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:42.093152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765" event={"ID":"78d77088-3b22-4472-a801-0aa478c3906b","Type":"ContainerDied","Data":"4e0f05c44e609fa2494749f9b3527937cc611991c7a49bbce76d4f22551e2f2b"}
Jul 22 15:52:43.237101 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:43.237071    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:43.268871 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:43.268842    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d77088-3b22-4472-a801-0aa478c3906b-maas-api-service-ca\") pod \"78d77088-3b22-4472-a801-0aa478c3906b\" (UID: \"78d77088-3b22-4472-a801-0aa478c3906b\") "
Jul 22 15:52:43.269188 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:43.269164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78d77088-3b22-4472-a801-0aa478c3906b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78d77088-3b22-4472-a801-0aa478c3906b" (UID: "78d77088-3b22-4472-a801-0aa478c3906b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:52:43.370553 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:43.370444    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d77088-3b22-4472-a801-0aa478c3906b-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:52:44.105645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:44.105614    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"
Jul 22 15:52:44.105645 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:44.105619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765" event={"ID":"78d77088-3b22-4472-a801-0aa478c3906b","Type":"ContainerDied","Data":"9854ec12af323dce0dfe0bc198c5dfe950fa31d5da0a63463186ac978913cb87"}
Jul 22 15:52:44.105871 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:52:44.105668    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9854ec12af323dce0dfe0bc198c5dfe950fa31d5da0a63463186ac978913cb87"
Jul 22 15:53:00.508795 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:00.508754    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"]
Jul 22 15:53:00.513064 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:00.513035    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-063391-b6765"]
Jul 22 15:53:01.021062 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:01.021027    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d77088-3b22-4472-a801-0aa478c3906b" path="/var/lib/kubelet/pods/78d77088-3b22-4472-a801-0aa478c3906b/volumes"
Jul 22 15:53:07.316117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.316084    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"]
Jul 22 15:53:07.316557 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.316543    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d77088-3b22-4472-a801-0aa478c3906b" containerName="revoke-keys"
Jul 22 15:53:07.316557 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.316558    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d77088-3b22-4472-a801-0aa478c3906b" containerName="revoke-keys"
Jul 22 15:53:07.316648 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.316638    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d77088-3b22-4472-a801-0aa478c3906b" containerName="revoke-keys"
Jul 22 15:53:07.319955 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.319935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:07.323408 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.323386    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:53:07.323555 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.323457    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 15:53:07.331505 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.331418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"]
Jul 22 15:53:07.417254 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.417219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f85521fe-a878-4c86-a8e6-649d692b2f36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f08216-gkssk\" (UID: \"f85521fe-a878-4c86-a8e6-649d692b2f36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:07.518106 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.518074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f85521fe-a878-4c86-a8e6-649d692b2f36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f08216-gkssk\" (UID: \"f85521fe-a878-4c86-a8e6-649d692b2f36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:07.518714 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.518696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f85521fe-a878-4c86-a8e6-649d692b2f36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f08216-gkssk\" (UID: \"f85521fe-a878-4c86-a8e6-649d692b2f36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:07.632381 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.632286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:07.774863 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:07.774840    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"]
Jul 22 15:53:07.777741 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:53:07.777633    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85521fe_a878_4c86_a8e6_649d692b2f36.slice/crio-3cce646575f5f36d22e2ddb5c0e83c410b4b3e09a4b5be91cf4731c7f4368a14 WatchSource:0}: Error finding container 3cce646575f5f36d22e2ddb5c0e83c410b4b3e09a4b5be91cf4731c7f4368a14: Status 404 returned error can't find the container with id 3cce646575f5f36d22e2ddb5c0e83c410b4b3e09a4b5be91cf4731c7f4368a14
Jul 22 15:53:08.234562 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:08.234452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk" event={"ID":"f85521fe-a878-4c86-a8e6-649d692b2f36","Type":"ContainerStarted","Data":"ef418029484f01ab63151c1808cf6fbaa27cc822e6678bd07c6cafbadb995f78"}
Jul 22 15:53:08.234562 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:08.234524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk" event={"ID":"f85521fe-a878-4c86-a8e6-649d692b2f36","Type":"ContainerStarted","Data":"3cce646575f5f36d22e2ddb5c0e83c410b4b3e09a4b5be91cf4731c7f4368a14"}
Jul 22 15:53:08.252147 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:08.252098    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk" podStartSLOduration=1.252079607 podStartE2EDuration="1.252079607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:53:08.251121068 +0000 UTC m=+2397.693204517" watchObservedRunningTime="2026-07-22 15:53:08.252079607 +0000 UTC m=+2397.694163054"
Jul 22 15:53:09.240845 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:09.240810    2580 generic.go:356] "Generic (PLEG): container finished" podID="f85521fe-a878-4c86-a8e6-649d692b2f36" containerID="ef418029484f01ab63151c1808cf6fbaa27cc822e6678bd07c6cafbadb995f78" exitCode=0
Jul 22 15:53:09.241282 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:09.240898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk" event={"ID":"f85521fe-a878-4c86-a8e6-649d692b2f36","Type":"ContainerDied","Data":"ef418029484f01ab63151c1808cf6fbaa27cc822e6678bd07c6cafbadb995f78"}
Jul 22 15:53:10.387680 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:10.387653    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:10.446086 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:10.446045    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f85521fe-a878-4c86-a8e6-649d692b2f36-maas-api-service-ca\") pod \"f85521fe-a878-4c86-a8e6-649d692b2f36\" (UID: \"f85521fe-a878-4c86-a8e6-649d692b2f36\") "
Jul 22 15:53:10.446405 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:10.446383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f85521fe-a878-4c86-a8e6-649d692b2f36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f85521fe-a878-4c86-a8e6-649d692b2f36" (UID: "f85521fe-a878-4c86-a8e6-649d692b2f36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:53:10.547574 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:10.547452    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f85521fe-a878-4c86-a8e6-649d692b2f36-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:53:11.252663 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:11.252629    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"
Jul 22 15:53:11.252838 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:11.252637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk" event={"ID":"f85521fe-a878-4c86-a8e6-649d692b2f36","Type":"ContainerDied","Data":"3cce646575f5f36d22e2ddb5c0e83c410b4b3e09a4b5be91cf4731c7f4368a14"}
Jul 22 15:53:11.252838 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:11.252775    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3cce646575f5f36d22e2ddb5c0e83c410b4b3e09a4b5be91cf4731c7f4368a14"
Jul 22 15:53:11.272620 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:11.272592    2580 scope.go:117] "RemoveContainer" containerID="65ccd39c1509c52ebacb843268568bd2ea4db9b08d30715ff3f9149a0f64cdca"
Jul 22 15:53:27.438607 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:27.438559    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"]
Jul 22 15:53:27.440612 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:27.440589    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f08216-gkssk"]
Jul 22 15:53:29.022263 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:53:29.022216    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f85521fe-a878-4c86-a8e6-649d692b2f36" path="/var/lib/kubelet/pods/f85521fe-a878-4c86-a8e6-649d692b2f36/volumes"
Jul 22 15:54:11.122739 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.122705    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"]
Jul 22 15:54:11.123313 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.123291    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f85521fe-a878-4c86-a8e6-649d692b2f36" containerName="revoke-keys"
Jul 22 15:54:11.123395 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.123317    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f85521fe-a878-4c86-a8e6-649d692b2f36" containerName="revoke-keys"
Jul 22 15:54:11.123505 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.123477    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f85521fe-a878-4c86-a8e6-649d692b2f36" containerName="revoke-keys"
Jul 22 15:54:11.126765 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.126741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.130967 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.130940    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-290ac7\"/\"default-dockercfg-p2ffq\""
Jul 22 15:54:11.131449 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.131429    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-290ac7\"/\"kube-root-ca.crt\""
Jul 22 15:54:11.132041 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.132025    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-290ac7\"/\"auth-test-model-290ac7-kserve-self-signed-certs\""
Jul 22 15:54:11.132421 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.132405    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-290ac7\"/\"openshift-service-ca.crt\""
Jul 22 15:54:11.140258 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.140230    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"]
Jul 22 15:54:11.241719 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.241680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tls-certs\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.241719 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.241722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-model-cache\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.241939 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.241859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-dshm\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.241982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.241936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jknpc\" (UniqueName: \"kubernetes.io/projected/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-kube-api-access-jknpc\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.241982 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.241967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-home\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.242047 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.241982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tmp-dir\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.292785 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.292753    2580 scope.go:117] "RemoveContainer" containerID="6f4b374dcf92f9a720b0e64a6ee7db68702db6df6da01bc7f3b629b45dc9d9b1"
Jul 22 15:54:11.343250 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jknpc\" (UniqueName: \"kubernetes.io/projected/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-kube-api-access-jknpc\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343429 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-home\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343429 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tmp-dir\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343544 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tls-certs\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343544 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-model-cache\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343655 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-dshm\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343731 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-home\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343793 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tmp-dir\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.343857 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.343841    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-model-cache\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.346038 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.346015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-dshm\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.346325 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.346305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tls-certs\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.355767 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.355741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jknpc\" (UniqueName: \"kubernetes.io/projected/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-kube-api-access-jknpc\") pod \"auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") " pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.441535 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.441401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:11.587430 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:11.587402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"]
Jul 22 15:54:11.589703 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:54:11.589666    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d5cc1f9_4aa7_4833_8781_7b5ff7da89b6.slice/crio-3e0a61fbc1768790a3da2f99eacd3229890a3670f9b2f532615d901618ce1f44 WatchSource:0}: Error finding container 3e0a61fbc1768790a3da2f99eacd3229890a3670f9b2f532615d901618ce1f44: Status 404 returned error can't find the container with id 3e0a61fbc1768790a3da2f99eacd3229890a3670f9b2f532615d901618ce1f44
Jul 22 15:54:12.587935 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:12.587900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv" event={"ID":"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6","Type":"ContainerStarted","Data":"e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12"}
Jul 22 15:54:12.588344 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:12.587944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv" event={"ID":"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6","Type":"ContainerStarted","Data":"3e0a61fbc1768790a3da2f99eacd3229890a3670f9b2f532615d901618ce1f44"}
Jul 22 15:54:12.607184 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:12.607124    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv" podStartSLOduration=1.607107944 podStartE2EDuration="1.607107944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:54:12.606124851 +0000 UTC m=+2462.048208300" watchObservedRunningTime="2026-07-22 15:54:12.607107944 +0000 UTC m=+2462.049191391"
Jul 22 15:54:21.442093 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:21.442047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:21.442638 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:21.442110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:21.454869 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:21.454838    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:54:21.644991 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:54:21.644960    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:57:14.028126 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:14.028083    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"]
Jul 22 15:57:27.628443 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.628409    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"]
Jul 22 15:57:27.632060 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.632038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.634956 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.634922    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-290ac7\"/\"sub-test-model-290ac7-kserve-self-signed-certs\""
Jul 22 15:57:27.643015 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.642987    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"]
Jul 22 15:57:27.727402 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.727366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-home\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.727618 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.727412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-257rv\" (UniqueName: \"kubernetes.io/projected/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-kube-api-access-257rv\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.727618 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.727438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tmp-dir\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.727618 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.727550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tls-certs\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.727618 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.727578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-model-cache\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.727758 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.727620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-dshm\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.828611 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.828561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-home\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.828611 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.828612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-257rv\" (UniqueName: \"kubernetes.io/projected/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-kube-api-access-257rv\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.828860 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.828636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tmp-dir\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.828860 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.828677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tls-certs\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.828860 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.828693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-model-cache\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.828860 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.828735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-dshm\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.829082 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.829063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tmp-dir\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.829138 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.829062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-home\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.829226 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.829201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-model-cache\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.831202 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.831182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-dshm\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.831449 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.831432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tls-certs\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.838733 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.838688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-257rv\" (UniqueName: \"kubernetes.io/projected/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-kube-api-access-257rv\") pod \"sub-test-model-290ac7-kserve-5774955ddc-5bwlh\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") " pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:27.942850 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:27.942759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:28.088751 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:28.088724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"]
Jul 22 15:57:28.090769 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:57:28.090742    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa22a812_5d1d_4cf9_8987_33e838f3aaa3.slice/crio-3a326ece9cfc5974edd1a343587b076eb5326c89996cd708994c27c8d1dfd1bf WatchSource:0}: Error finding container 3a326ece9cfc5974edd1a343587b076eb5326c89996cd708994c27c8d1dfd1bf: Status 404 returned error can't find the container with id 3a326ece9cfc5974edd1a343587b076eb5326c89996cd708994c27c8d1dfd1bf
Jul 22 15:57:28.613561 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:28.613522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh" event={"ID":"aa22a812-5d1d-4cf9-8987-33e838f3aaa3","Type":"ContainerStarted","Data":"15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56"}
Jul 22 15:57:28.613561 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:28.613568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh" event={"ID":"aa22a812-5d1d-4cf9-8987-33e838f3aaa3","Type":"ContainerStarted","Data":"3a326ece9cfc5974edd1a343587b076eb5326c89996cd708994c27c8d1dfd1bf"}
Jul 22 15:57:28.633818 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:28.633750    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh" podStartSLOduration=1.633729956 podStartE2EDuration="1.633729956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:57:28.633693867 +0000 UTC m=+2658.075777315" watchObservedRunningTime="2026-07-22 15:57:28.633729956 +0000 UTC m=+2658.075813404"
Jul 22 15:57:29.049144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.049093    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv" podUID="3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" containerName="main" containerID="cri-o://e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12" gracePeriod=45
Jul 22 15:57:29.306429 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.306358    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:57:29.445820 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.445782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tmp-dir\") pod \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") "
Jul 22 15:57:29.446027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.445843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jknpc\" (UniqueName: \"kubernetes.io/projected/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-kube-api-access-jknpc\") pod \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") "
Jul 22 15:57:29.446027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.445891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-home\") pod \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") "
Jul 22 15:57:29.446027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.445952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-dshm\") pod \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") "
Jul 22 15:57:29.446027 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-model-cache\") pod \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") "
Jul 22 15:57:29.446238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446034    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tls-certs\") pod \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\" (UID: \"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6\") "
Jul 22 15:57:29.446238 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" (UID: "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:57:29.446338 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446280    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-home" (OuterVolumeSpecName: "home") pod "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" (UID: "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:57:29.446338 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446301    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" (UID: "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:57:29.446519 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446476    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tmp-dir\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:57:29.446588 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446523    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-home\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:57:29.446588 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.446533    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-model-cache\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:57:29.448455 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.448436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-dshm" (OuterVolumeSpecName: "dshm") pod "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" (UID: "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:57:29.448455 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.448425    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" (UID: "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:57:29.448608 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.448548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-kube-api-access-jknpc" (OuterVolumeSpecName: "kube-api-access-jknpc") pod "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" (UID: "3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6"). InnerVolumeSpecName "kube-api-access-jknpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:57:29.547543 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.547465    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jknpc\" (UniqueName: \"kubernetes.io/projected/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-kube-api-access-jknpc\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:57:29.547543 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.547535    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-dshm\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:57:29.547543 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.547545    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6-tls-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:57:29.620839 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.620741    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" containerID="e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12" exitCode=0
Jul 22 15:57:29.620839 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.620784    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"
Jul 22 15:57:29.620839 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.620828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv" event={"ID":"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6","Type":"ContainerDied","Data":"e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12"}
Jul 22 15:57:29.621117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.620874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv" event={"ID":"3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6","Type":"ContainerDied","Data":"3e0a61fbc1768790a3da2f99eacd3229890a3670f9b2f532615d901618ce1f44"}
Jul 22 15:57:29.621117 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.620891    2580 scope.go:117] "RemoveContainer" containerID="e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12"
Jul 22 15:57:29.630424 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.630406    2580 scope.go:117] "RemoveContainer" containerID="e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12"
Jul 22 15:57:29.630756 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:57:29.630739    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12\": container with ID starting with e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12 not found: ID does not exist" containerID="e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12"
Jul 22 15:57:29.630820 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.630766    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12"} err="failed to get container status \"e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12\": rpc error: code = NotFound desc = could not find container \"e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12\": container with ID starting with e92491f4a91322f63d9cb36bb9c785a5830f5ee239d6be8124d875a9e0860d12 not found: ID does not exist"
Jul 22 15:57:29.648822 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.648539    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"]
Jul 22 15:57:29.650138 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:29.650115    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/auth-test-model-290ac7-kserve-6d4cd784f5-sn2zv"]
Jul 22 15:57:31.021230 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:31.021187    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" path="/var/lib/kubelet/pods/3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6/volumes"
Jul 22 15:57:37.943766 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:37.943718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:37.944192 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:37.943793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:37.956333 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:37.956304    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:38.681058 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:38.681023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:57:53.032076 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.032042    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"]
Jul 22 15:57:53.034746 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.032689    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" containerName="main"
Jul 22 15:57:53.034746 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.032705    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" containerName="main"
Jul 22 15:57:53.034746 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.032773    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5cc1f9-4aa7-4833-8781-7b5ff7da89b6" containerName="main"
Jul 22 15:57:53.035718 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.035697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.038439 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.038415    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"kube-root-ca.crt\""
Jul 22 15:57:53.039450 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.039426    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"default-dockercfg-96mc2\""
Jul 22 15:57:53.039593 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.039446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"openshift-service-ca.crt\""
Jul 22 15:57:53.039593 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.039552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"sub-test-model-9911b9-kserve-self-signed-certs\""
Jul 22 15:57:53.045672 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.045646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"]
Jul 22 15:57:53.171894 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.171857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-home\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.172092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.171902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-model-cache\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.172092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.171924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rrrk\" (UniqueName: \"kubernetes.io/projected/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-kube-api-access-5rrrk\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.172092 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.172060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tls-certs\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.172271 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.172093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-dshm\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.172271 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.172196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tmp-dir\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.273717 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.273676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tmp-dir\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.273928 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.273735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-home\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.273928 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.273765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-model-cache\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.273928 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.273783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rrrk\" (UniqueName: \"kubernetes.io/projected/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-kube-api-access-5rrrk\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.274108 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.273930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tls-certs\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.274108 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.273973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-dshm\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.274245 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.274221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-home\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.274245 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.274239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-model-cache\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.274396 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.274238    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tmp-dir\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.276383 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.276364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-dshm\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.276687 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.276667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tls-certs\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.284700 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.284633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rrrk\" (UniqueName: \"kubernetes.io/projected/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-kube-api-access-5rrrk\") pod \"sub-test-model-9911b9-kserve-6b6b79d985-pljjh\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") " pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.346913 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.346880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:57:53.489468 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.489441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"]
Jul 22 15:57:53.491446 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:57:53.491409    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e9702c4_c9ba_4bfa_beb7_884b2d4e6446.slice/crio-8e72326bad850b62da22e945c55c32b0af94d94f6f0e420360f6d44a6554cb2f WatchSource:0}: Error finding container 8e72326bad850b62da22e945c55c32b0af94d94f6f0e420360f6d44a6554cb2f: Status 404 returned error can't find the container with id 8e72326bad850b62da22e945c55c32b0af94d94f6f0e420360f6d44a6554cb2f
Jul 22 15:57:53.757577 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.757529    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh" event={"ID":"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446","Type":"ContainerStarted","Data":"bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3"}
Jul 22 15:57:53.757577 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.757582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh" event={"ID":"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446","Type":"ContainerStarted","Data":"8e72326bad850b62da22e945c55c32b0af94d94f6f0e420360f6d44a6554cb2f"}
Jul 22 15:57:53.782225 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:57:53.782169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh" podStartSLOduration=0.782152949 podStartE2EDuration="782.152949ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:57:53.780260612 +0000 UTC m=+2683.222344059" watchObservedRunningTime="2026-07-22 15:57:53.782152949 +0000 UTC m=+2683.224236395"
Jul 22 15:58:03.347316 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:03.347216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:58:03.347316 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:03.347293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:58:03.359510 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:03.359463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:58:03.821960 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:03.821924    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:58:11.338295 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:11.338263    2580 scope.go:117] "RemoveContainer" containerID="5900c58aae10b1a94c01572b0665618e216efc2026808767a7d2214628792d2a"
Jul 22 15:58:11.347133 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:11.347111    2580 scope.go:117] "RemoveContainer" containerID="3ec0ae196791caa7a8315afd1ea6b59eeb91d9854acfc949bd3c34f465f90c1d"
Jul 22 15:58:39.482449 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:39.482398    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"]
Jul 22 15:58:40.251310 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:40.251271    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"]
Jul 22 15:58:54.503136 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.503083    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh" podUID="aa22a812-5d1d-4cf9-8987-33e838f3aaa3" containerName="main" containerID="cri-o://15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56" gracePeriod=45
Jul 22 15:58:54.762215 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.762151    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:58:54.774030 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.773999    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tls-certs\") pod \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") "
Jul 22 15:58:54.774217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774092    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tmp-dir\") pod \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") "
Jul 22 15:58:54.774217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774129    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-home\") pod \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") "
Jul 22 15:58:54.774217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774151    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-257rv\" (UniqueName: \"kubernetes.io/projected/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-kube-api-access-257rv\") pod \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") "
Jul 22 15:58:54.774217 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774172    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-model-cache\") pod \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") "
Jul 22 15:58:54.774434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-dshm\") pod \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\" (UID: \"aa22a812-5d1d-4cf9-8987-33e838f3aaa3\") "
Jul 22 15:58:54.774434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774375    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa22a812-5d1d-4cf9-8987-33e838f3aaa3" (UID: "aa22a812-5d1d-4cf9-8987-33e838f3aaa3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:54.774434 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-home" (OuterVolumeSpecName: "home") pod "aa22a812-5d1d-4cf9-8987-33e838f3aaa3" (UID: "aa22a812-5d1d-4cf9-8987-33e838f3aaa3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:54.774598 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774461    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa22a812-5d1d-4cf9-8987-33e838f3aaa3" (UID: "aa22a812-5d1d-4cf9-8987-33e838f3aaa3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:54.774657 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774623    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tmp-dir\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:54.774657 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774638    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-home\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:54.774657 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.774648    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-model-cache\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:54.776403 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.776380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-dshm" (OuterVolumeSpecName: "dshm") pod "aa22a812-5d1d-4cf9-8987-33e838f3aaa3" (UID: "aa22a812-5d1d-4cf9-8987-33e838f3aaa3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:54.776534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.776390    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa22a812-5d1d-4cf9-8987-33e838f3aaa3" (UID: "aa22a812-5d1d-4cf9-8987-33e838f3aaa3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:58:54.776534 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.776439    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-kube-api-access-257rv" (OuterVolumeSpecName: "kube-api-access-257rv") pod "aa22a812-5d1d-4cf9-8987-33e838f3aaa3" (UID: "aa22a812-5d1d-4cf9-8987-33e838f3aaa3"). InnerVolumeSpecName "kube-api-access-257rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:58:54.875865 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.875826    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-dshm\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:54.875865 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.875856    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-tls-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:54.875865 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:54.875868    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-257rv\" (UniqueName: \"kubernetes.io/projected/aa22a812-5d1d-4cf9-8987-33e838f3aaa3-kube-api-access-257rv\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:55.074970 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.074937    2580 generic.go:356] "Generic (PLEG): container finished" podID="aa22a812-5d1d-4cf9-8987-33e838f3aaa3" containerID="15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56" exitCode=0
Jul 22 15:58:55.075154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.074984    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"
Jul 22 15:58:55.075154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.075005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh" event={"ID":"aa22a812-5d1d-4cf9-8987-33e838f3aaa3","Type":"ContainerDied","Data":"15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56"}
Jul 22 15:58:55.075154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.075047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh" event={"ID":"aa22a812-5d1d-4cf9-8987-33e838f3aaa3","Type":"ContainerDied","Data":"3a326ece9cfc5974edd1a343587b076eb5326c89996cd708994c27c8d1dfd1bf"}
Jul 22 15:58:55.075154 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.075069    2580 scope.go:117] "RemoveContainer" containerID="15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56"
Jul 22 15:58:55.084065 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.084044    2580 scope.go:117] "RemoveContainer" containerID="15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56"
Jul 22 15:58:55.084378 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:58:55.084358    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56\": container with ID starting with 15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56 not found: ID does not exist" containerID="15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56"
Jul 22 15:58:55.084445 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.084391    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56"} err="failed to get container status \"15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56\": rpc error: code = NotFound desc = could not find container \"15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56\": container with ID starting with 15c0187b189724b0eaa58cb35f3206632067d113d930daf03e5c3287b234da56 not found: ID does not exist"
Jul 22 15:58:55.093153 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.093125    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"]
Jul 22 15:58:55.096807 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.096780    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-290ac7/sub-test-model-290ac7-kserve-5774955ddc-5bwlh"]
Jul 22 15:58:55.273163 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.273122    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh" podUID="4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" containerName="main" containerID="cri-o://bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3" gracePeriod=45
Jul 22 15:58:55.545193 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.545128    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:58:55.582222 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tls-certs\") pod \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") "
Jul 22 15:58:55.582412 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582282    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rrrk\" (UniqueName: \"kubernetes.io/projected/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-kube-api-access-5rrrk\") pod \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") "
Jul 22 15:58:55.582412 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-dshm\") pod \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") "
Jul 22 15:58:55.582412 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582345    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-home\") pod \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") "
Jul 22 15:58:55.582412 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582377    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tmp-dir\") pod \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") "
Jul 22 15:58:55.582667 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582445    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-model-cache\") pod \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\" (UID: \"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446\") "
Jul 22 15:58:55.582667 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582652    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-home" (OuterVolumeSpecName: "home") pod "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" (UID: "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:55.582779 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" (UID: "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:55.582819 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582794    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-model-cache\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:55.582819 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582813    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-home\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:55.582906 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.582891    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" (UID: "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:55.584790 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.584764    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-kube-api-access-5rrrk" (OuterVolumeSpecName: "kube-api-access-5rrrk") pod "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" (UID: "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446"). InnerVolumeSpecName "kube-api-access-5rrrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:58:55.584902 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.584854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" (UID: "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:58:55.584945 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.584893    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-dshm" (OuterVolumeSpecName: "dshm") pod "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" (UID: "4e9702c4-c9ba-4bfa-beb7-884b2d4e6446"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:58:55.683696 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.683652    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rrrk\" (UniqueName: \"kubernetes.io/projected/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-kube-api-access-5rrrk\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:55.683696 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.683687    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-dshm\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:55.683696 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.683697    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tmp-dir\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:55.683696 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:55.683706    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446-tls-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:58:56.082144 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.082108    2580 generic.go:356] "Generic (PLEG): container finished" podID="4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" containerID="bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3" exitCode=0
Jul 22 15:58:56.082358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.082173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh" event={"ID":"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446","Type":"ContainerDied","Data":"bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3"}
Jul 22 15:58:56.082358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.082180    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"
Jul 22 15:58:56.082358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.082204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh" event={"ID":"4e9702c4-c9ba-4bfa-beb7-884b2d4e6446","Type":"ContainerDied","Data":"8e72326bad850b62da22e945c55c32b0af94d94f6f0e420360f6d44a6554cb2f"}
Jul 22 15:58:56.082358 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.082222    2580 scope.go:117] "RemoveContainer" containerID="bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3"
Jul 22 15:58:56.092376 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.092341    2580 scope.go:117] "RemoveContainer" containerID="bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3"
Jul 22 15:58:56.092694 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:58:56.092675    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3\": container with ID starting with bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3 not found: ID does not exist" containerID="bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3"
Jul 22 15:58:56.092761 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.092703    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3"} err="failed to get container status \"bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3\": rpc error: code = NotFound desc = could not find container \"bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3\": container with ID starting with bf0efe7b89bc5374993fee5470946634a44760bbd01abde81b47afba5631e8a3 not found: ID does not exist"
Jul 22 15:58:56.107002 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.106979    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"]
Jul 22 15:58:56.108950 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:56.108928    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/sub-test-model-9911b9-kserve-6b6b79d985-pljjh"]
Jul 22 15:58:57.022570 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:57.022539    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" path="/var/lib/kubelet/pods/4e9702c4-c9ba-4bfa-beb7-884b2d4e6446/volumes"
Jul 22 15:58:57.022975 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:58:57.022946    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa22a812-5d1d-4cf9-8987-33e838f3aaa3" path="/var/lib/kubelet/pods/aa22a812-5d1d-4cf9-8987-33e838f3aaa3/volumes"
Jul 22 15:59:04.875235 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875189    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"]
Jul 22 15:59:04.875792 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875770    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" containerName="main"
Jul 22 15:59:04.875868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875796    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" containerName="main"
Jul 22 15:59:04.875868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875844    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa22a812-5d1d-4cf9-8987-33e838f3aaa3" containerName="main"
Jul 22 15:59:04.875868 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875853    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa22a812-5d1d-4cf9-8987-33e838f3aaa3" containerName="main"
Jul 22 15:59:04.876014 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875962    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9702c4-c9ba-4bfa-beb7-884b2d4e6446" containerName="main"
Jul 22 15:59:04.876014 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.875979    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa22a812-5d1d-4cf9-8987-33e838f3aaa3" containerName="main"
Jul 22 15:59:04.880651 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.880629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:04.886764 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.886730    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"kube-root-ca.crt\""
Jul 22 15:59:04.886764 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.886763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"openshift-service-ca.crt\""
Jul 22 15:59:04.890478 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.890457    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"default-dockercfg-96mc2\""
Jul 22 15:59:04.890767 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.890749    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-9911b9\"/\"rate-test-model-9911b9-kserve-self-signed-certs\""
Jul 22 15:59:04.900532 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.900502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"]
Jul 22 15:59:04.970463 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.970424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxbcs\" (UniqueName: \"kubernetes.io/projected/9bcc0e74-8a24-44a2-8104-4143999cb3cf-kube-api-access-dxbcs\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:04.970691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.970500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tls-certs\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:04.970691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.970552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-model-cache\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:04.970691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.970581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-home\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:04.970691 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.970608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-dshm\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:04.970883 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:04.970694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tmp-dir\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072023 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.071986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-model-cache\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-home\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-dshm\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tmp-dir\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxbcs\" (UniqueName: \"kubernetes.io/projected/9bcc0e74-8a24-44a2-8104-4143999cb3cf-kube-api-access-dxbcs\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072227 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tls-certs\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072525 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-model-cache\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072525 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-home\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.072674 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.072552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tmp-dir\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.074715 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.074684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-dshm\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.074917 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.074900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tls-certs\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.087539 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.087484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxbcs\" (UniqueName: \"kubernetes.io/projected/9bcc0e74-8a24-44a2-8104-4143999cb3cf-kube-api-access-dxbcs\") pod \"rate-test-model-9911b9-kserve-6ddc4d6d87-whd55\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") " pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.190563 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.190465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:05.347631 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:05.347599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"]
Jul 22 15:59:05.351829 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:59:05.351799    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bcc0e74_8a24_44a2_8104_4143999cb3cf.slice/crio-abecf9d7219cd9cd2f806d6ceabeed1a589ff43dbf9c715f13c19b7805b4fc3a WatchSource:0}: Error finding container abecf9d7219cd9cd2f806d6ceabeed1a589ff43dbf9c715f13c19b7805b4fc3a: Status 404 returned error can't find the container with id abecf9d7219cd9cd2f806d6ceabeed1a589ff43dbf9c715f13c19b7805b4fc3a
Jul 22 15:59:06.143581 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:06.143532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55" event={"ID":"9bcc0e74-8a24-44a2-8104-4143999cb3cf","Type":"ContainerStarted","Data":"0801d77c7918b060372951dfda42c9098415df7ebc2288d101f37df059d30ae8"}
Jul 22 15:59:06.143983 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:06.143591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55" event={"ID":"9bcc0e74-8a24-44a2-8104-4143999cb3cf","Type":"ContainerStarted","Data":"abecf9d7219cd9cd2f806d6ceabeed1a589ff43dbf9c715f13c19b7805b4fc3a"}
Jul 22 15:59:06.180041 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:06.179985    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55" podStartSLOduration=2.1799681140000002 podStartE2EDuration="2.179968114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:59:06.178980488 +0000 UTC m=+2755.621063935" watchObservedRunningTime="2026-07-22 15:59:06.179968114 +0000 UTC m=+2755.622051561"
Jul 22 15:59:11.376582 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:11.376545    2580 scope.go:117] "RemoveContainer" containerID="ef418029484f01ab63151c1808cf6fbaa27cc822e6678bd07c6cafbadb995f78"
Jul 22 15:59:11.385257 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:11.385232    2580 scope.go:117] "RemoveContainer" containerID="4e0f05c44e609fa2494749f9b3527937cc611991c7a49bbce76d4f22551e2f2b"
Jul 22 15:59:15.190693 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:15.190659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:15.191116 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:15.190744    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:15.204014 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:15.203978    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:16.207150 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:16.207117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 15:59:53.251032 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:53.250944    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"]
Jul 22 15:59:54.231425 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.231389    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7680-140559985019568"]
Jul 22 15:59:54.234954 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.234933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:54.237786 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.237760    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ngwkz\""
Jul 22 15:59:54.242784 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.242760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985019568"]
Jul 22 15:59:54.351132 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.351098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvznj\" (UniqueName: \"kubernetes.io/projected/e8828e4b-a553-4474-93ae-d5df18f3308b-kube-api-access-bvznj\") pod \"test-curl-7680-140559985019568\" (UID: \"e8828e4b-a553-4474-93ae-d5df18f3308b\") " pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:54.452102 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.452055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvznj\" (UniqueName: \"kubernetes.io/projected/e8828e4b-a553-4474-93ae-d5df18f3308b-kube-api-access-bvznj\") pod \"test-curl-7680-140559985019568\" (UID: \"e8828e4b-a553-4474-93ae-d5df18f3308b\") " pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:54.462224 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.462189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvznj\" (UniqueName: \"kubernetes.io/projected/e8828e4b-a553-4474-93ae-d5df18f3308b-kube-api-access-bvznj\") pod \"test-curl-7680-140559985019568\" (UID: \"e8828e4b-a553-4474-93ae-d5df18f3308b\") " pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:54.545318 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.545274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:54.685864 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.685832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985019568"]
Jul 22 15:59:54.688085 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:59:54.688050    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8828e4b_a553_4474_93ae_d5df18f3308b.slice/crio-99eea7870f457271198deb1894e3343685e43167472a3f854eb1f337002add76 WatchSource:0}: Error finding container 99eea7870f457271198deb1894e3343685e43167472a3f854eb1f337002add76: Status 404 returned error can't find the container with id 99eea7870f457271198deb1894e3343685e43167472a3f854eb1f337002add76
Jul 22 15:59:54.689557 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:54.689536    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:59:55.391931 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:55.391898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985019568" event={"ID":"e8828e4b-a553-4474-93ae-d5df18f3308b","Type":"ContainerStarted","Data":"99eea7870f457271198deb1894e3343685e43167472a3f854eb1f337002add76"}
Jul 22 15:59:56.398592 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:56.398555    2580 generic.go:356] "Generic (PLEG): container finished" podID="e8828e4b-a553-4474-93ae-d5df18f3308b" containerID="9940571b61796acdc2157c464b600087145ccac472589a0e16f23ad886848f52" exitCode=0
Jul 22 15:59:56.398998 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:56.398642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985019568" event={"ID":"e8828e4b-a553-4474-93ae-d5df18f3308b","Type":"ContainerDied","Data":"9940571b61796acdc2157c464b600087145ccac472589a0e16f23ad886848f52"}
Jul 22 15:59:57.540013 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.539985    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:57.581438 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.581402    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7680-140559985019568_e8828e4b-a553-4474-93ae-d5df18f3308b/test-curl-7680-140559985019568/0.log"
Jul 22 15:59:57.611290 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.611251    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7680-140559985019568"]
Jul 22 15:59:57.617385 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.617361    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7680-140559985019568"]
Jul 22 15:59:57.685140 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.685048    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvznj\" (UniqueName: \"kubernetes.io/projected/e8828e4b-a553-4474-93ae-d5df18f3308b-kube-api-access-bvznj\") pod \"e8828e4b-a553-4474-93ae-d5df18f3308b\" (UID: \"e8828e4b-a553-4474-93ae-d5df18f3308b\") "
Jul 22 15:59:57.687342 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.687313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8828e4b-a553-4474-93ae-d5df18f3308b-kube-api-access-bvznj" (OuterVolumeSpecName: "kube-api-access-bvznj") pod "e8828e4b-a553-4474-93ae-d5df18f3308b" (UID: "e8828e4b-a553-4474-93ae-d5df18f3308b"). InnerVolumeSpecName "kube-api-access-bvznj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:59:57.739443 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.739409    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7680-140559984857648"]
Jul 22 15:59:57.739834 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.739819    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8828e4b-a553-4474-93ae-d5df18f3308b" containerName="test-curl-7680-140559985019568"
Jul 22 15:59:57.739893 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.739836    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8828e4b-a553-4474-93ae-d5df18f3308b" containerName="test-curl-7680-140559985019568"
Jul 22 15:59:57.739933 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.739916    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8828e4b-a553-4474-93ae-d5df18f3308b" containerName="test-curl-7680-140559985019568"
Jul 22 15:59:57.743026 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.743008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 15:59:57.751847 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.751825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559984857648"]
Jul 22 15:59:57.786006 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.785961    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvznj\" (UniqueName: \"kubernetes.io/projected/e8828e4b-a553-4474-93ae-d5df18f3308b-kube-api-access-bvznj\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 15:59:57.887484 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.887438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9bb\" (UniqueName: \"kubernetes.io/projected/0cdbe36a-d288-41c2-b46f-f2b83676d11b-kube-api-access-gv9bb\") pod \"test-curl-7680-140559984857648\" (UID: \"0cdbe36a-d288-41c2-b46f-f2b83676d11b\") " pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 15:59:57.988658 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.988551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9bb\" (UniqueName: \"kubernetes.io/projected/0cdbe36a-d288-41c2-b46f-f2b83676d11b-kube-api-access-gv9bb\") pod \"test-curl-7680-140559984857648\" (UID: \"0cdbe36a-d288-41c2-b46f-f2b83676d11b\") " pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 15:59:57.997084 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:57.997057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9bb\" (UniqueName: \"kubernetes.io/projected/0cdbe36a-d288-41c2-b46f-f2b83676d11b-kube-api-access-gv9bb\") pod \"test-curl-7680-140559984857648\" (UID: \"0cdbe36a-d288-41c2-b46f-f2b83676d11b\") " pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 15:59:58.052935 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:58.052900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 15:59:58.195094 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:58.195070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559984857648"]
Jul 22 15:59:58.196891 ip-10-0-143-231 kubenswrapper[2580]: W0722 15:59:58.196868    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cdbe36a_d288_41c2_b46f_f2b83676d11b.slice/crio-7763cc3a7e730d7c8d08884897518b150585528c56f655ed9df7da3a6e3d5911 WatchSource:0}: Error finding container 7763cc3a7e730d7c8d08884897518b150585528c56f655ed9df7da3a6e3d5911: Status 404 returned error can't find the container with id 7763cc3a7e730d7c8d08884897518b150585528c56f655ed9df7da3a6e3d5911
Jul 22 15:59:58.409934 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:58.409907    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:58.410156 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:58.409951    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99eea7870f457271198deb1894e3343685e43167472a3f854eb1f337002add76"
Jul 22 15:59:58.411158 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:58.411132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559984857648" event={"ID":"0cdbe36a-d288-41c2-b46f-f2b83676d11b","Type":"ContainerStarted","Data":"7763cc3a7e730d7c8d08884897518b150585528c56f655ed9df7da3a6e3d5911"}
Jul 22 15:59:58.420762 ip-10-0-143-231 kubenswrapper[2580]: E0722 15:59:58.420732    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7680-140559985019568\" is forbidden: User \"system:node:ip-10-0-143-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-231.ec2.internal' and this object" podUID="e8828e4b-a553-4474-93ae-d5df18f3308b" pod="opendatahub/test-curl-7680-140559985019568"
Jul 22 15:59:59.021773 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:59.021739    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8828e4b-a553-4474-93ae-d5df18f3308b" path="/var/lib/kubelet/pods/e8828e4b-a553-4474-93ae-d5df18f3308b/volumes"
Jul 22 15:59:59.418682 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:59.418564    2580 generic.go:356] "Generic (PLEG): container finished" podID="0cdbe36a-d288-41c2-b46f-f2b83676d11b" containerID="906a89bd95bc67ad64606385c7bf22c5d28dc1050ad4ab8b8fdd8a53daf4dbf3" exitCode=0
Jul 22 15:59:59.418682 ip-10-0-143-231 kubenswrapper[2580]: I0722 15:59:59.418651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559984857648" event={"ID":"0cdbe36a-d288-41c2-b46f-f2b83676d11b","Type":"ContainerDied","Data":"906a89bd95bc67ad64606385c7bf22c5d28dc1050ad4ab8b8fdd8a53daf4dbf3"}
Jul 22 16:00:00.153474 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.153375    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"]
Jul 22 16:00:00.157369 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.157348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"
Jul 22 16:00:00.171306 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.171252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 16:00:00.173573 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.173547    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"]
Jul 22 16:00:00.177815 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.177791    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"]
Jul 22 16:00:00.182145 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.179155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"
Jul 22 16:00:00.182145 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.179661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"
Jul 22 16:00:00.184647 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.184622    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"]
Jul 22 16:00:00.184999 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.184830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"
Jul 22 16:00:00.184999 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.184831    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"]
Jul 22 16:00:00.186459 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.185979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"]
Jul 22 16:00:00.195217 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.195190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"
Jul 22 16:00:00.229746 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.229713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"
Jul 22 16:00:00.357199 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.357173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"]
Jul 22 16:00:00.359453 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:00.359427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc925f699_2319_4e16_9c00_f9e7499790df.slice/crio-c57ac4004a24162a6ba4a6b3cb7659f56dace563078871b0fa248239fd8de24f WatchSource:0}: Error finding container c57ac4004a24162a6ba4a6b3cb7659f56dace563078871b0fa248239fd8de24f: Status 404 returned error can't find the container with id c57ac4004a24162a6ba4a6b3cb7659f56dace563078871b0fa248239fd8de24f
Jul 22 16:00:00.416319 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.416292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"]
Jul 22 16:00:00.418242 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:00.418212    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode869a481_a2e1_444b_b65f_21873b64d195.slice/crio-c1c7587c337048429df0c4de470b3ce5cefcca237e647647dbacd33665de2728 WatchSource:0}: Error finding container c1c7587c337048429df0c4de470b3ce5cefcca237e647647dbacd33665de2728: Status 404 returned error can't find the container with id c1c7587c337048429df0c4de470b3ce5cefcca237e647647dbacd33665de2728
Jul 22 16:00:00.424686 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.424662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt" event={"ID":"e869a481-a2e1-444b-b65f-21873b64d195","Type":"ContainerStarted","Data":"c1c7587c337048429df0c4de470b3ce5cefcca237e647647dbacd33665de2728"}
Jul 22 16:00:00.425945 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.425924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs" event={"ID":"c925f699-2319-4e16-9c00-f9e7499790df","Type":"ContainerStarted","Data":"c57ac4004a24162a6ba4a6b3cb7659f56dace563078871b0fa248239fd8de24f"}
Jul 22 16:00:00.555890 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.555857    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 16:00:00.580442 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.580413    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7680-140559984857648_0cdbe36a-d288-41c2-b46f-f2b83676d11b/test-curl-7680-140559984857648/0.log"
Jul 22 16:00:00.592416 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.592383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"]
Jul 22 16:00:00.594787 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:00.594760    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f17ca8_8fab_4c4a_8ef0_43691e00430f.slice/crio-713d6d1471bb9ee7cbb0e0c1e34ec5aaec78d87aa8adb6f6dc6c829b5e9f0405 WatchSource:0}: Error finding container 713d6d1471bb9ee7cbb0e0c1e34ec5aaec78d87aa8adb6f6dc6c829b5e9f0405: Status 404 returned error can't find the container with id 713d6d1471bb9ee7cbb0e0c1e34ec5aaec78d87aa8adb6f6dc6c829b5e9f0405
Jul 22 16:00:00.605642 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.605618    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7680-140559984857648"]
Jul 22 16:00:00.610549 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.610526    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7680-140559984857648"]
Jul 22 16:00:00.616379 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.616357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv9bb\" (UniqueName: \"kubernetes.io/projected/0cdbe36a-d288-41c2-b46f-f2b83676d11b-kube-api-access-gv9bb\") pod \"0cdbe36a-d288-41c2-b46f-f2b83676d11b\" (UID: \"0cdbe36a-d288-41c2-b46f-f2b83676d11b\") "
Jul 22 16:00:00.618476 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.618448    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cdbe36a-d288-41c2-b46f-f2b83676d11b-kube-api-access-gv9bb" (OuterVolumeSpecName: "kube-api-access-gv9bb") pod "0cdbe36a-d288-41c2-b46f-f2b83676d11b" (UID: "0cdbe36a-d288-41c2-b46f-f2b83676d11b"). InnerVolumeSpecName "kube-api-access-gv9bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:00:00.718143 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.718047    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv9bb\" (UniqueName: \"kubernetes.io/projected/0cdbe36a-d288-41c2-b46f-f2b83676d11b-kube-api-access-gv9bb\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:00.771964 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.771934    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:00.772398 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.772384    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cdbe36a-d288-41c2-b46f-f2b83676d11b" containerName="test-curl-7680-140559984857648"
Jul 22 16:00:00.772438 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.772400    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cdbe36a-d288-41c2-b46f-f2b83676d11b" containerName="test-curl-7680-140559984857648"
Jul 22 16:00:00.772501 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.772477    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cdbe36a-d288-41c2-b46f-f2b83676d11b" containerName="test-curl-7680-140559984857648"
Jul 22 16:00:00.775520 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.775472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:00.784828 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.784801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:00.919880 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:00.919841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t4ht\" (UniqueName: \"kubernetes.io/projected/e9ade034-af37-41c4-a7c1-3f21f93d84c7-kube-api-access-7t4ht\") pod \"test-curl-7680-140559985062944\" (UID: \"e9ade034-af37-41c4-a7c1-3f21f93d84c7\") " pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:01.020662 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.020630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t4ht\" (UniqueName: \"kubernetes.io/projected/e9ade034-af37-41c4-a7c1-3f21f93d84c7-kube-api-access-7t4ht\") pod \"test-curl-7680-140559985062944\" (UID: \"e9ade034-af37-41c4-a7c1-3f21f93d84c7\") " pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:01.021714 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.021688    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cdbe36a-d288-41c2-b46f-f2b83676d11b" path="/var/lib/kubelet/pods/0cdbe36a-d288-41c2-b46f-f2b83676d11b/volumes"
Jul 22 16:00:01.032555 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.032530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t4ht\" (UniqueName: \"kubernetes.io/projected/e9ade034-af37-41c4-a7c1-3f21f93d84c7-kube-api-access-7t4ht\") pod \"test-curl-7680-140559985062944\" (UID: \"e9ade034-af37-41c4-a7c1-3f21f93d84c7\") " pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:01.085147 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.085103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:01.231608 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.231442    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:01.234098 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:01.234070    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9ade034_af37_41c4_a7c1_3f21f93d84c7.slice/crio-97e16c5e4fcbc6392fb6ed8c4e193da2bccedeee92ad34b54478ec08c8ecab16 WatchSource:0}: Error finding container 97e16c5e4fcbc6392fb6ed8c4e193da2bccedeee92ad34b54478ec08c8ecab16: Status 404 returned error can't find the container with id 97e16c5e4fcbc6392fb6ed8c4e193da2bccedeee92ad34b54478ec08c8ecab16
Jul 22 16:00:01.434037 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.434004    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559984857648"
Jul 22 16:00:01.434248 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.434013    2580 scope.go:117] "RemoveContainer" containerID="906a89bd95bc67ad64606385c7bf22c5d28dc1050ad4ab8b8fdd8a53daf4dbf3"
Jul 22 16:00:01.435915 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.435884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt" event={"ID":"70f17ca8-8fab-4c4a-8ef0-43691e00430f","Type":"ContainerStarted","Data":"12c289d3099c2ddf65f4fa5e3433573d063d69f0c15446110372e7a4c4869778"}
Jul 22 16:00:01.436033 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.435945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt" event={"ID":"70f17ca8-8fab-4c4a-8ef0-43691e00430f","Type":"ContainerStarted","Data":"713d6d1471bb9ee7cbb0e0c1e34ec5aaec78d87aa8adb6f6dc6c829b5e9f0405"}
Jul 22 16:00:01.437609 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.437587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt" event={"ID":"e869a481-a2e1-444b-b65f-21873b64d195","Type":"ContainerStarted","Data":"78237cbc27992a5ddd719cc35a8fc1b3eeb71e3952dabf4bc56e302f27243221"}
Jul 22 16:00:01.439306 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.439282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs" event={"ID":"c925f699-2319-4e16-9c00-f9e7499790df","Type":"ContainerStarted","Data":"222e29f5b588757eed4491254fc0f531622c49cdcad106e25ccfaaf7fab06c97"}
Jul 22 16:00:01.440582 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.440558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985062944" event={"ID":"e9ade034-af37-41c4-a7c1-3f21f93d84c7","Type":"ContainerStarted","Data":"97e16c5e4fcbc6392fb6ed8c4e193da2bccedeee92ad34b54478ec08c8ecab16"}
Jul 22 16:00:01.455597 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.455539    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt" podStartSLOduration=1.455483202 podStartE2EDuration="1.455483202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:00:01.452989285 +0000 UTC m=+2810.895072732" watchObservedRunningTime="2026-07-22 16:00:01.455483202 +0000 UTC m=+2810.897566649"
Jul 22 16:00:01.472878 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.472834    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt" podStartSLOduration=1.472817461 podStartE2EDuration="1.472817461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:00:01.471736386 +0000 UTC m=+2810.913819835" watchObservedRunningTime="2026-07-22 16:00:01.472817461 +0000 UTC m=+2810.914900907"
Jul 22 16:00:01.490570 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:01.490522    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs" podStartSLOduration=1.49050751 podStartE2EDuration="1.49050751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:00:01.488910115 +0000 UTC m=+2810.930993563" watchObservedRunningTime="2026-07-22 16:00:01.49050751 +0000 UTC m=+2810.932590948"
Jul 22 16:00:02.447221 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.447185    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9ade034-af37-41c4-a7c1-3f21f93d84c7" containerID="04864a15e6938e2b32fbc7916cc8b44ba759fd4eea437f15d80144ed45fbac03" exitCode=0
Jul 22 16:00:02.447687 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.447274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985062944" event={"ID":"e9ade034-af37-41c4-a7c1-3f21f93d84c7","Type":"ContainerDied","Data":"04864a15e6938e2b32fbc7916cc8b44ba759fd4eea437f15d80144ed45fbac03"}
Jul 22 16:00:02.449320 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.449299    2580 generic.go:356] "Generic (PLEG): container finished" podID="70f17ca8-8fab-4c4a-8ef0-43691e00430f" containerID="12c289d3099c2ddf65f4fa5e3433573d063d69f0c15446110372e7a4c4869778" exitCode=0
Jul 22 16:00:02.449445 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.449382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt" event={"ID":"70f17ca8-8fab-4c4a-8ef0-43691e00430f","Type":"ContainerDied","Data":"12c289d3099c2ddf65f4fa5e3433573d063d69f0c15446110372e7a4c4869778"}
Jul 22 16:00:02.450651 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.450633    2580 generic.go:356] "Generic (PLEG): container finished" podID="e869a481-a2e1-444b-b65f-21873b64d195" containerID="78237cbc27992a5ddd719cc35a8fc1b3eeb71e3952dabf4bc56e302f27243221" exitCode=0
Jul 22 16:00:02.450742 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.450663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt" event={"ID":"e869a481-a2e1-444b-b65f-21873b64d195","Type":"ContainerDied","Data":"78237cbc27992a5ddd719cc35a8fc1b3eeb71e3952dabf4bc56e302f27243221"}
Jul 22 16:00:02.452184 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.452165    2580 generic.go:356] "Generic (PLEG): container finished" podID="c925f699-2319-4e16-9c00-f9e7499790df" containerID="222e29f5b588757eed4491254fc0f531622c49cdcad106e25ccfaaf7fab06c97" exitCode=0
Jul 22 16:00:02.452265 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:02.452232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs" event={"ID":"c925f699-2319-4e16-9c00-f9e7499790df","Type":"ContainerDied","Data":"222e29f5b588757eed4491254fc0f531622c49cdcad106e25ccfaaf7fab06c97"}
Jul 22 16:00:03.614882 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.614854    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"
Jul 22 16:00:03.672758 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.672732    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:03.687105 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.687073    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"
Jul 22 16:00:03.691044 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.691017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"
Jul 22 16:00:03.692137 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.692111    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7680-140559985062944_e9ade034-af37-41c4-a7c1-3f21f93d84c7/test-curl-7680-140559985062944/0.log"
Jul 22 16:00:03.715661 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.715580    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:03.723781 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.723747    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:03.745624 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.745593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t4ht\" (UniqueName: \"kubernetes.io/projected/e9ade034-af37-41c4-a7c1-3f21f93d84c7-kube-api-access-7t4ht\") pod \"e9ade034-af37-41c4-a7c1-3f21f93d84c7\" (UID: \"e9ade034-af37-41c4-a7c1-3f21f93d84c7\") "
Jul 22 16:00:03.747856 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.747834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9ade034-af37-41c4-a7c1-3f21f93d84c7-kube-api-access-7t4ht" (OuterVolumeSpecName: "kube-api-access-7t4ht") pod "e9ade034-af37-41c4-a7c1-3f21f93d84c7" (UID: "e9ade034-af37-41c4-a7c1-3f21f93d84c7"). InnerVolumeSpecName "kube-api-access-7t4ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:00:03.838415 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838378    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:03.838813 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838798    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f17ca8-8fab-4c4a-8ef0-43691e00430f" containerName="cleanup"
Jul 22 16:00:03.838865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838815    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f17ca8-8fab-4c4a-8ef0-43691e00430f" containerName="cleanup"
Jul 22 16:00:03.838865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838828    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ade034-af37-41c4-a7c1-3f21f93d84c7" containerName="test-curl-7680-140559985062944"
Jul 22 16:00:03.838865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838836    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ade034-af37-41c4-a7c1-3f21f93d84c7" containerName="test-curl-7680-140559985062944"
Jul 22 16:00:03.838865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838859    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925f699-2319-4e16-9c00-f9e7499790df" containerName="cleanup"
Jul 22 16:00:03.838865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838865    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925f699-2319-4e16-9c00-f9e7499790df" containerName="cleanup"
Jul 22 16:00:03.839021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838872    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e869a481-a2e1-444b-b65f-21873b64d195" containerName="cleanup"
Jul 22 16:00:03.839021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838877    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e869a481-a2e1-444b-b65f-21873b64d195" containerName="cleanup"
Jul 22 16:00:03.839021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838937    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9ade034-af37-41c4-a7c1-3f21f93d84c7" containerName="test-curl-7680-140559985062944"
Jul 22 16:00:03.839021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838947    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f17ca8-8fab-4c4a-8ef0-43691e00430f" containerName="cleanup"
Jul 22 16:00:03.839021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838955    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925f699-2319-4e16-9c00-f9e7499790df" containerName="cleanup"
Jul 22 16:00:03.839021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.838961    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e869a481-a2e1-444b-b65f-21873b64d195" containerName="cleanup"
Jul 22 16:00:03.842121 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.842104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:03.846633 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.846602    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t4ht\" (UniqueName: \"kubernetes.io/projected/e9ade034-af37-41c4-a7c1-3f21f93d84c7-kube-api-access-7t4ht\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:03.850826 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.850801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:03.947600 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:03.947559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj8vl\" (UniqueName: \"kubernetes.io/projected/04b88db3-9238-4663-9f47-c33ebe8fbca8-kube-api-access-cj8vl\") pod \"test-curl-7680-140559985062944\" (UID: \"04b88db3-9238-4663-9f47-c33ebe8fbca8\") " pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:04.048565 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.048519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj8vl\" (UniqueName: \"kubernetes.io/projected/04b88db3-9238-4663-9f47-c33ebe8fbca8-kube-api-access-cj8vl\") pod \"test-curl-7680-140559985062944\" (UID: \"04b88db3-9238-4663-9f47-c33ebe8fbca8\") " pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:04.057629 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.057597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj8vl\" (UniqueName: \"kubernetes.io/projected/04b88db3-9238-4663-9f47-c33ebe8fbca8-kube-api-access-cj8vl\") pod \"test-curl-7680-140559985062944\" (UID: \"04b88db3-9238-4663-9f47-c33ebe8fbca8\") " pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:04.152587 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.152550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:04.293027 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.293001    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:04.295653 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:04.295624    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04b88db3_9238_4663_9f47_c33ebe8fbca8.slice/crio-0cb29edd9c53631891e17e81279ae60c7d2c0bc6e30adad49b2b3e568eed0cc9 WatchSource:0}: Error finding container 0cb29edd9c53631891e17e81279ae60c7d2c0bc6e30adad49b2b3e568eed0cc9: Status 404 returned error can't find the container with id 0cb29edd9c53631891e17e81279ae60c7d2c0bc6e30adad49b2b3e568eed0cc9
Jul 22 16:00:04.466296 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.466178    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"
Jul 22 16:00:04.466296 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.466190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt" event={"ID":"70f17ca8-8fab-4c4a-8ef0-43691e00430f","Type":"ContainerDied","Data":"713d6d1471bb9ee7cbb0e0c1e34ec5aaec78d87aa8adb6f6dc6c829b5e9f0405"}
Jul 22 16:00:04.466296 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.466246    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="713d6d1471bb9ee7cbb0e0c1e34ec5aaec78d87aa8adb6f6dc6c829b5e9f0405"
Jul 22 16:00:04.467918 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.467893    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt"
Jul 22 16:00:04.467918 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.467906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745600-dn4gt" event={"ID":"e869a481-a2e1-444b-b65f-21873b64d195","Type":"ContainerDied","Data":"c1c7587c337048429df0c4de470b3ce5cefcca237e647647dbacd33665de2728"}
Jul 22 16:00:04.468312 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.467955    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1c7587c337048429df0c4de470b3ce5cefcca237e647647dbacd33665de2728"
Jul 22 16:00:04.475138 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.469911    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"
Jul 22 16:00:04.475138 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.470016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs" event={"ID":"c925f699-2319-4e16-9c00-f9e7499790df","Type":"ContainerDied","Data":"c57ac4004a24162a6ba4a6b3cb7659f56dace563078871b0fa248239fd8de24f"}
Jul 22 16:00:04.475138 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.470058    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c57ac4004a24162a6ba4a6b3cb7659f56dace563078871b0fa248239fd8de24f"
Jul 22 16:00:04.475138 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.472135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985062944" event={"ID":"04b88db3-9238-4663-9f47-c33ebe8fbca8","Type":"ContainerStarted","Data":"0cb29edd9c53631891e17e81279ae60c7d2c0bc6e30adad49b2b3e568eed0cc9"}
Jul 22 16:00:04.475138 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.474257    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97e16c5e4fcbc6392fb6ed8c4e193da2bccedeee92ad34b54478ec08c8ecab16"
Jul 22 16:00:04.475138 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.474318    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:04.495089 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:04.495054    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7680-140559985062944" oldPodUID="e9ade034-af37-41c4-a7c1-3f21f93d84c7" podUID="04b88db3-9238-4663-9f47-c33ebe8fbca8"
Jul 22 16:00:05.022563 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:05.022529    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9ade034-af37-41c4-a7c1-3f21f93d84c7" path="/var/lib/kubelet/pods/e9ade034-af37-41c4-a7c1-3f21f93d84c7/volumes"
Jul 22 16:00:08.271610 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.271562    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55" podUID="9bcc0e74-8a24-44a2-8104-4143999cb3cf" containerName="main" containerID="cri-o://0801d77c7918b060372951dfda42c9098415df7ebc2288d101f37df059d30ae8" gracePeriod=45
Jul 22 16:00:08.497911 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.497866    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bcc0e74-8a24-44a2-8104-4143999cb3cf" containerID="0801d77c7918b060372951dfda42c9098415df7ebc2288d101f37df059d30ae8" exitCode=0
Jul 22 16:00:08.498063 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.497934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55" event={"ID":"9bcc0e74-8a24-44a2-8104-4143999cb3cf","Type":"ContainerDied","Data":"0801d77c7918b060372951dfda42c9098415df7ebc2288d101f37df059d30ae8"}
Jul 22 16:00:08.527541 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.527444    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 16:00:08.691812 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.691769    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-model-cache\") pod \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") "
Jul 22 16:00:08.691997 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.691843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-home\") pod \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") "
Jul 22 16:00:08.691997 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.691920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tls-certs\") pod \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") "
Jul 22 16:00:08.691997 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.691960    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tmp-dir\") pod \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") "
Jul 22 16:00:08.691997 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.691989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-dshm\") pod \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") "
Jul 22 16:00:08.692221 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692042    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxbcs\" (UniqueName: \"kubernetes.io/projected/9bcc0e74-8a24-44a2-8104-4143999cb3cf-kube-api-access-dxbcs\") pod \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\" (UID: \"9bcc0e74-8a24-44a2-8104-4143999cb3cf\") "
Jul 22 16:00:08.692221 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-home" (OuterVolumeSpecName: "home") pod "9bcc0e74-8a24-44a2-8104-4143999cb3cf" (UID: "9bcc0e74-8a24-44a2-8104-4143999cb3cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:00:08.692221 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692102    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "9bcc0e74-8a24-44a2-8104-4143999cb3cf" (UID: "9bcc0e74-8a24-44a2-8104-4143999cb3cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:00:08.692349 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692240    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9bcc0e74-8a24-44a2-8104-4143999cb3cf" (UID: "9bcc0e74-8a24-44a2-8104-4143999cb3cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:00:08.692408 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692377    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tmp-dir\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:08.692408 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692392    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-model-cache\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:08.692408 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.692401    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-home\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:08.694221 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.694199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-dshm" (OuterVolumeSpecName: "dshm") pod "9bcc0e74-8a24-44a2-8104-4143999cb3cf" (UID: "9bcc0e74-8a24-44a2-8104-4143999cb3cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:00:08.694338 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.694244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bcc0e74-8a24-44a2-8104-4143999cb3cf-kube-api-access-dxbcs" (OuterVolumeSpecName: "kube-api-access-dxbcs") pod "9bcc0e74-8a24-44a2-8104-4143999cb3cf" (UID: "9bcc0e74-8a24-44a2-8104-4143999cb3cf"). InnerVolumeSpecName "kube-api-access-dxbcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:00:08.694338 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.694258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9bcc0e74-8a24-44a2-8104-4143999cb3cf" (UID: "9bcc0e74-8a24-44a2-8104-4143999cb3cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:00:08.793468 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.793374    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcc0e74-8a24-44a2-8104-4143999cb3cf-tls-certs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:08.793468 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.793407    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcc0e74-8a24-44a2-8104-4143999cb3cf-dshm\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:08.793468 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:08.793416    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxbcs\" (UniqueName: \"kubernetes.io/projected/9bcc0e74-8a24-44a2-8104-4143999cb3cf-kube-api-access-dxbcs\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:09.504367 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:09.504340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"
Jul 22 16:00:09.504865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:09.504383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55" event={"ID":"9bcc0e74-8a24-44a2-8104-4143999cb3cf","Type":"ContainerDied","Data":"abecf9d7219cd9cd2f806d6ceabeed1a589ff43dbf9c715f13c19b7805b4fc3a"}
Jul 22 16:00:09.504865 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:09.504438    2580 scope.go:117] "RemoveContainer" containerID="0801d77c7918b060372951dfda42c9098415df7ebc2288d101f37df059d30ae8"
Jul 22 16:00:09.524121 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:09.524086    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"]
Jul 22 16:00:09.529373 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:09.529345    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-9911b9/rate-test-model-9911b9-kserve-6ddc4d6d87-whd55"]
Jul 22 16:00:10.512708 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:10.512672    2580 generic.go:356] "Generic (PLEG): container finished" podID="04b88db3-9238-4663-9f47-c33ebe8fbca8" containerID="7d430c0d33489c6e840eb5602bc617395379415fae46671791a31b77449f91c5" exitCode=0
Jul 22 16:00:10.513156 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:10.512762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985062944" event={"ID":"04b88db3-9238-4663-9f47-c33ebe8fbca8","Type":"ContainerDied","Data":"7d430c0d33489c6e840eb5602bc617395379415fae46671791a31b77449f91c5"}
Jul 22 16:00:11.027412 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.027376    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bcc0e74-8a24-44a2-8104-4143999cb3cf" path="/var/lib/kubelet/pods/9bcc0e74-8a24-44a2-8104-4143999cb3cf/volumes"
Jul 22 16:00:11.663363 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.663330    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:11.685937 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.685856    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7680-140559985062944_04b88db3-9238-4663-9f47-c33ebe8fbca8/test-curl-7680-140559985062944/0.log"
Jul 22 16:00:11.710121 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.710089    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:11.713939 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.713910    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7680-140559985062944"]
Jul 22 16:00:11.820136 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.820096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj8vl\" (UniqueName: \"kubernetes.io/projected/04b88db3-9238-4663-9f47-c33ebe8fbca8-kube-api-access-cj8vl\") pod \"04b88db3-9238-4663-9f47-c33ebe8fbca8\" (UID: \"04b88db3-9238-4663-9f47-c33ebe8fbca8\") "
Jul 22 16:00:11.822337 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.822304    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b88db3-9238-4663-9f47-c33ebe8fbca8-kube-api-access-cj8vl" (OuterVolumeSpecName: "kube-api-access-cj8vl") pod "04b88db3-9238-4663-9f47-c33ebe8fbca8" (UID: "04b88db3-9238-4663-9f47-c33ebe8fbca8"). InnerVolumeSpecName "kube-api-access-cj8vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:00:11.841169 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841137    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:11.841545 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841531    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bcc0e74-8a24-44a2-8104-4143999cb3cf" containerName="main"
Jul 22 16:00:11.841599 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841546    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bcc0e74-8a24-44a2-8104-4143999cb3cf" containerName="main"
Jul 22 16:00:11.841599 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841576    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b88db3-9238-4663-9f47-c33ebe8fbca8" containerName="test-curl-7680-140559985062944"
Jul 22 16:00:11.841599 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841582    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b88db3-9238-4663-9f47-c33ebe8fbca8" containerName="test-curl-7680-140559985062944"
Jul 22 16:00:11.841713 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841647    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bcc0e74-8a24-44a2-8104-4143999cb3cf" containerName="main"
Jul 22 16:00:11.841713 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.841658    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b88db3-9238-4663-9f47-c33ebe8fbca8" containerName="test-curl-7680-140559985062944"
Jul 22 16:00:11.844707 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.844688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:11.852901 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.852871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:11.921393 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:11.921357    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj8vl\" (UniqueName: \"kubernetes.io/projected/04b88db3-9238-4663-9f47-c33ebe8fbca8-kube-api-access-cj8vl\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:12.022076 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.022028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8wsc\" (UniqueName: \"kubernetes.io/projected/e788f9da-4493-4989-802c-36631d5b37fb-kube-api-access-q8wsc\") pod \"test-curl-7680-140559985063952\" (UID: \"e788f9da-4493-4989-802c-36631d5b37fb\") " pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:12.123478 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.123437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8wsc\" (UniqueName: \"kubernetes.io/projected/e788f9da-4493-4989-802c-36631d5b37fb-kube-api-access-q8wsc\") pod \"test-curl-7680-140559985063952\" (UID: \"e788f9da-4493-4989-802c-36631d5b37fb\") " pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:12.132687 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.132654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8wsc\" (UniqueName: \"kubernetes.io/projected/e788f9da-4493-4989-802c-36631d5b37fb-kube-api-access-q8wsc\") pod \"test-curl-7680-140559985063952\" (UID: \"e788f9da-4493-4989-802c-36631d5b37fb\") " pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:12.154623 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.154585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:12.323071 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.323031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:12.323988 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:12.323962    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode788f9da_4493_4989_802c_36631d5b37fb.slice/crio-0d1205f796337ce37774a8e9b56b6ba18f00a1a476701d588d8c2a0d92e0b194 WatchSource:0}: Error finding container 0d1205f796337ce37774a8e9b56b6ba18f00a1a476701d588d8c2a0d92e0b194: Status 404 returned error can't find the container with id 0d1205f796337ce37774a8e9b56b6ba18f00a1a476701d588d8c2a0d92e0b194
Jul 22 16:00:12.525669 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.525637    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:12.525669 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.525663    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cb29edd9c53631891e17e81279ae60c7d2c0bc6e30adad49b2b3e568eed0cc9"
Jul 22 16:00:12.526886 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:12.526861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985063952" event={"ID":"e788f9da-4493-4989-802c-36631d5b37fb","Type":"ContainerStarted","Data":"0d1205f796337ce37774a8e9b56b6ba18f00a1a476701d588d8c2a0d92e0b194"}
Jul 22 16:00:12.538917 ip-10-0-143-231 kubenswrapper[2580]: E0722 16:00:12.538876    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7680-140559985062944\" is forbidden: User \"system:node:ip-10-0-143-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-231.ec2.internal' and this object" podUID="04b88db3-9238-4663-9f47-c33ebe8fbca8" pod="opendatahub/test-curl-7680-140559985062944"
Jul 22 16:00:13.021469 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:13.021423    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b88db3-9238-4663-9f47-c33ebe8fbca8" path="/var/lib/kubelet/pods/04b88db3-9238-4663-9f47-c33ebe8fbca8/volumes"
Jul 22 16:00:13.533216 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:13.533181    2580 generic.go:356] "Generic (PLEG): container finished" podID="e788f9da-4493-4989-802c-36631d5b37fb" containerID="534a928670c468018e5908c8f99d5e9dd6c50f2cb414470c2a9854820469219f" exitCode=0
Jul 22 16:00:13.533381 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:13.533258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985063952" event={"ID":"e788f9da-4493-4989-802c-36631d5b37fb","Type":"ContainerDied","Data":"534a928670c468018e5908c8f99d5e9dd6c50f2cb414470c2a9854820469219f"}
Jul 22 16:00:14.679838 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.679803    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:14.704750 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.704706    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7680-140559985063952_e788f9da-4493-4989-802c-36631d5b37fb/test-curl-7680-140559985063952/0.log"
Jul 22 16:00:14.731630 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.731594    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:14.734342 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.734313    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:14.856219 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.856127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8wsc\" (UniqueName: \"kubernetes.io/projected/e788f9da-4493-4989-802c-36631d5b37fb-kube-api-access-q8wsc\") pod \"e788f9da-4493-4989-802c-36631d5b37fb\" (UID: \"e788f9da-4493-4989-802c-36631d5b37fb\") "
Jul 22 16:00:14.858565 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.858529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e788f9da-4493-4989-802c-36631d5b37fb-kube-api-access-q8wsc" (OuterVolumeSpecName: "kube-api-access-q8wsc") pod "e788f9da-4493-4989-802c-36631d5b37fb" (UID: "e788f9da-4493-4989-802c-36631d5b37fb"). InnerVolumeSpecName "kube-api-access-q8wsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:00:14.859131 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.859109    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:14.859566 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.859552    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e788f9da-4493-4989-802c-36631d5b37fb" containerName="test-curl-7680-140559985063952"
Jul 22 16:00:14.859608 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.859567    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e788f9da-4493-4989-802c-36631d5b37fb" containerName="test-curl-7680-140559985063952"
Jul 22 16:00:14.859647 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.859629    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e788f9da-4493-4989-802c-36631d5b37fb" containerName="test-curl-7680-140559985063952"
Jul 22 16:00:14.862797 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.862781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:14.869602 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.869573    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:14.957205 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:14.957164    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8wsc\" (UniqueName: \"kubernetes.io/projected/e788f9da-4493-4989-802c-36631d5b37fb-kube-api-access-q8wsc\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:15.021605 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.021571    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e788f9da-4493-4989-802c-36631d5b37fb" path="/var/lib/kubelet/pods/e788f9da-4493-4989-802c-36631d5b37fb/volumes"
Jul 22 16:00:15.057949 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.057909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652hh\" (UniqueName: \"kubernetes.io/projected/e49753b2-1d09-4ef8-968c-c701e37aeeb1-kube-api-access-652hh\") pod \"test-curl-7680-140559985063952\" (UID: \"e49753b2-1d09-4ef8-968c-c701e37aeeb1\") " pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:15.158950 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.158855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652hh\" (UniqueName: \"kubernetes.io/projected/e49753b2-1d09-4ef8-968c-c701e37aeeb1-kube-api-access-652hh\") pod \"test-curl-7680-140559985063952\" (UID: \"e49753b2-1d09-4ef8-968c-c701e37aeeb1\") " pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:15.169092 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.169057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-652hh\" (UniqueName: \"kubernetes.io/projected/e49753b2-1d09-4ef8-968c-c701e37aeeb1-kube-api-access-652hh\") pod \"test-curl-7680-140559985063952\" (UID: \"e49753b2-1d09-4ef8-968c-c701e37aeeb1\") " pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:15.175947 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.175916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:15.315540 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.315513    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:15.317101 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:15.317076    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode49753b2_1d09_4ef8_968c_c701e37aeeb1.slice/crio-3e8a106f8ce2384ddf0e37c985b9ec9b36cd8145c6adb250d1a8061b634984e1 WatchSource:0}: Error finding container 3e8a106f8ce2384ddf0e37c985b9ec9b36cd8145c6adb250d1a8061b634984e1: Status 404 returned error can't find the container with id 3e8a106f8ce2384ddf0e37c985b9ec9b36cd8145c6adb250d1a8061b634984e1
Jul 22 16:00:15.552342 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.552264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985063952" event={"ID":"e49753b2-1d09-4ef8-968c-c701e37aeeb1","Type":"ContainerStarted","Data":"3e8a106f8ce2384ddf0e37c985b9ec9b36cd8145c6adb250d1a8061b634984e1"}
Jul 22 16:00:15.553600 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.553577    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:15.553713 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:15.553593    2580 scope.go:117] "RemoveContainer" containerID="534a928670c468018e5908c8f99d5e9dd6c50f2cb414470c2a9854820469219f"
Jul 22 16:00:16.560939 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:16.560904    2580 generic.go:356] "Generic (PLEG): container finished" podID="e49753b2-1d09-4ef8-968c-c701e37aeeb1" containerID="b4c29e930edb60e3d34d3691b04d5e7759e67969ede05374667098a922da99b6" exitCode=0
Jul 22 16:00:16.561424 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:16.560999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7680-140559985063952" event={"ID":"e49753b2-1d09-4ef8-968c-c701e37aeeb1","Type":"ContainerDied","Data":"b4c29e930edb60e3d34d3691b04d5e7759e67969ede05374667098a922da99b6"}
Jul 22 16:00:17.712354 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.712326    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:17.735591 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.735561    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7680-140559985063952_e49753b2-1d09-4ef8-968c-c701e37aeeb1/test-curl-7680-140559985063952/0.log"
Jul 22 16:00:17.761650 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.761612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:17.765776 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.765747    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7680-140559985063952"]
Jul 22 16:00:17.780964 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.780934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-652hh\" (UniqueName: \"kubernetes.io/projected/e49753b2-1d09-4ef8-968c-c701e37aeeb1-kube-api-access-652hh\") pod \"e49753b2-1d09-4ef8-968c-c701e37aeeb1\" (UID: \"e49753b2-1d09-4ef8-968c-c701e37aeeb1\") "
Jul 22 16:00:17.783181 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.783155    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49753b2-1d09-4ef8-968c-c701e37aeeb1-kube-api-access-652hh" (OuterVolumeSpecName: "kube-api-access-652hh") pod "e49753b2-1d09-4ef8-968c-c701e37aeeb1" (UID: "e49753b2-1d09-4ef8-968c-c701e37aeeb1"). InnerVolumeSpecName "kube-api-access-652hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:00:17.882136 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.882038    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-652hh\" (UniqueName: \"kubernetes.io/projected/e49753b2-1d09-4ef8-968c-c701e37aeeb1-kube-api-access-652hh\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:17.945028 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.944986    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"]
Jul 22 16:00:17.945432 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.945417    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49753b2-1d09-4ef8-968c-c701e37aeeb1" containerName="test-curl-7680-140559985063952"
Jul 22 16:00:17.945481 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.945434    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49753b2-1d09-4ef8-968c-c701e37aeeb1" containerName="test-curl-7680-140559985063952"
Jul 22 16:00:17.945546 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.945534    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49753b2-1d09-4ef8-968c-c701e37aeeb1" containerName="test-curl-7680-140559985063952"
Jul 22 16:00:17.949080 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.949055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:17.951784 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.951762    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 16:00:17.952011 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.951995    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:00:17.957623 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.957597    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"]
Jul 22 16:00:17.982677 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:17.982641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3deef5-d1ed-473b-acf1-7509051ff66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-290ac7-bv628\" (UID: \"7a3deef5-d1ed-473b-acf1-7509051ff66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:18.083945 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.083897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3deef5-d1ed-473b-acf1-7509051ff66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-290ac7-bv628\" (UID: \"7a3deef5-d1ed-473b-acf1-7509051ff66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:18.084619 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.084600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3deef5-d1ed-473b-acf1-7509051ff66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-290ac7-bv628\" (UID: \"7a3deef5-d1ed-473b-acf1-7509051ff66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:18.262918 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.262864    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:18.401479 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.401453    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"]
Jul 22 16:00:18.403764 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:18.403734    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3deef5_d1ed_473b_acf1_7509051ff66c.slice/crio-6cdb9bd281c9d2aba29972076032490177c169e65fc89d3b45847008f0e5c22d WatchSource:0}: Error finding container 6cdb9bd281c9d2aba29972076032490177c169e65fc89d3b45847008f0e5c22d: Status 404 returned error can't find the container with id 6cdb9bd281c9d2aba29972076032490177c169e65fc89d3b45847008f0e5c22d
Jul 22 16:00:18.577462 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.577436    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:18.577667 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.577436    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e8a106f8ce2384ddf0e37c985b9ec9b36cd8145c6adb250d1a8061b634984e1"
Jul 22 16:00:18.578764 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:18.578739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628" event={"ID":"7a3deef5-d1ed-473b-acf1-7509051ff66c","Type":"ContainerStarted","Data":"6cdb9bd281c9d2aba29972076032490177c169e65fc89d3b45847008f0e5c22d"}
Jul 22 16:00:18.587678 ip-10-0-143-231 kubenswrapper[2580]: E0722 16:00:18.587648    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7680-140559985063952\" is forbidden: User \"system:node:ip-10-0-143-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-231.ec2.internal' and this object" podUID="e49753b2-1d09-4ef8-968c-c701e37aeeb1" pod="opendatahub/test-curl-7680-140559985063952"
Jul 22 16:00:19.021471 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:19.021434    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49753b2-1d09-4ef8-968c-c701e37aeeb1" path="/var/lib/kubelet/pods/e49753b2-1d09-4ef8-968c-c701e37aeeb1/volumes"
Jul 22 16:00:19.585363 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:19.585326    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a3deef5-d1ed-473b-acf1-7509051ff66c" containerID="b08061533c745f7ba4b8bfc1db09eb48408cbad662d2bd67d1406e51ce0fa2ab" exitCode=0
Jul 22 16:00:19.585577 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:19.585417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628" event={"ID":"7a3deef5-d1ed-473b-acf1-7509051ff66c","Type":"ContainerDied","Data":"b08061533c745f7ba4b8bfc1db09eb48408cbad662d2bd67d1406e51ce0fa2ab"}
Jul 22 16:00:20.726979 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:20.726954    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:20.811351 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:20.811314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3deef5-d1ed-473b-acf1-7509051ff66c-maas-api-service-ca\") pod \"7a3deef5-d1ed-473b-acf1-7509051ff66c\" (UID: \"7a3deef5-d1ed-473b-acf1-7509051ff66c\") "
Jul 22 16:00:20.811734 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:20.811712    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a3deef5-d1ed-473b-acf1-7509051ff66c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a3deef5-d1ed-473b-acf1-7509051ff66c" (UID: "7a3deef5-d1ed-473b-acf1-7509051ff66c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:00:20.912557 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:20.912464    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a3deef5-d1ed-473b-acf1-7509051ff66c-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:21.596839 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:21.596805    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"
Jul 22 16:00:21.597042 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:21.596798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628" event={"ID":"7a3deef5-d1ed-473b-acf1-7509051ff66c","Type":"ContainerDied","Data":"6cdb9bd281c9d2aba29972076032490177c169e65fc89d3b45847008f0e5c22d"}
Jul 22 16:00:21.597042 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:21.596948    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cdb9bd281c9d2aba29972076032490177c169e65fc89d3b45847008f0e5c22d"
Jul 22 16:00:28.403436 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:28.403395    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"]
Jul 22 16:00:28.408227 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:28.408199    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-290ac7-29745600-hg9kt"]
Jul 22 16:00:29.021083 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:29.021043    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70f17ca8-8fab-4c4a-8ef0-43691e00430f" path="/var/lib/kubelet/pods/70f17ca8-8fab-4c4a-8ef0-43691e00430f/volumes"
Jul 22 16:00:33.034880 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:33.034839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"]
Jul 22 16:00:33.037694 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:33.037669    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-290ac7-bv628"]
Jul 22 16:00:35.022132 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:35.022096    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3deef5-d1ed-473b-acf1-7509051ff66c" path="/var/lib/kubelet/pods/7a3deef5-d1ed-473b-acf1-7509051ff66c/volumes"
Jul 22 16:00:40.074307 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.074269    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"]
Jul 22 16:00:40.074735 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.074725    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3deef5-d1ed-473b-acf1-7509051ff66c" containerName="revoke-keys"
Jul 22 16:00:40.074783 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.074737    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3deef5-d1ed-473b-acf1-7509051ff66c" containerName="revoke-keys"
Jul 22 16:00:40.074821 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.074809    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3deef5-d1ed-473b-acf1-7509051ff66c" containerName="revoke-keys"
Jul 22 16:00:40.078163 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.078143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:40.080778 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.080751    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:00:40.080913 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.080750    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j4mpg\""
Jul 22 16:00:40.085417 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.085387    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"]
Jul 22 16:00:40.200529 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.200470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05ec3e29-f84b-4b86-a731-466cda0284f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs\" (UID: \"05ec3e29-f84b-4b86-a731-466cda0284f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:40.301744 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.301712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05ec3e29-f84b-4b86-a731-466cda0284f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs\" (UID: \"05ec3e29-f84b-4b86-a731-466cda0284f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:40.302341 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.302321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05ec3e29-f84b-4b86-a731-466cda0284f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs\" (UID: \"05ec3e29-f84b-4b86-a731-466cda0284f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:40.392407 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.392307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:40.538446 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.538415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"]
Jul 22 16:00:40.540508 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:00:40.540457    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ec3e29_f84b_4b86_a731_466cda0284f5.slice/crio-c815d9de4a38047b2a0ae06c8a8930afb6a56feafccca54243b41588dfcd52e9 WatchSource:0}: Error finding container c815d9de4a38047b2a0ae06c8a8930afb6a56feafccca54243b41588dfcd52e9: Status 404 returned error can't find the container with id c815d9de4a38047b2a0ae06c8a8930afb6a56feafccca54243b41588dfcd52e9
Jul 22 16:00:40.701921 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:40.701825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs" event={"ID":"05ec3e29-f84b-4b86-a731-466cda0284f5","Type":"ContainerStarted","Data":"c815d9de4a38047b2a0ae06c8a8930afb6a56feafccca54243b41588dfcd52e9"}
Jul 22 16:00:41.708253 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:41.708159    2580 generic.go:356] "Generic (PLEG): container finished" podID="05ec3e29-f84b-4b86-a731-466cda0284f5" containerID="80826c8a739ed1584941f9c6267ebe52cd5b889a3cdae8152f5f0b9868b11961" exitCode=0
Jul 22 16:00:41.708253 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:41.708215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs" event={"ID":"05ec3e29-f84b-4b86-a731-466cda0284f5","Type":"ContainerDied","Data":"80826c8a739ed1584941f9c6267ebe52cd5b889a3cdae8152f5f0b9868b11961"}
Jul 22 16:00:42.846625 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:42.846597    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:42.929733 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:42.929689    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05ec3e29-f84b-4b86-a731-466cda0284f5-maas-api-service-ca\") pod \"05ec3e29-f84b-4b86-a731-466cda0284f5\" (UID: \"05ec3e29-f84b-4b86-a731-466cda0284f5\") "
Jul 22 16:00:42.930082 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:42.930061    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05ec3e29-f84b-4b86-a731-466cda0284f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05ec3e29-f84b-4b86-a731-466cda0284f5" (UID: "05ec3e29-f84b-4b86-a731-466cda0284f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:00:43.031117 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:43.031087    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05ec3e29-f84b-4b86-a731-466cda0284f5-maas-api-service-ca\") on node \"ip-10-0-143-231.ec2.internal\" DevicePath \"\""
Jul 22 16:00:43.719802 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:43.719769    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"
Jul 22 16:00:43.719802 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:43.719796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs" event={"ID":"05ec3e29-f84b-4b86-a731-466cda0284f5","Type":"ContainerDied","Data":"c815d9de4a38047b2a0ae06c8a8930afb6a56feafccca54243b41588dfcd52e9"}
Jul 22 16:00:43.720036 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:43.719857    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c815d9de4a38047b2a0ae06c8a8930afb6a56feafccca54243b41588dfcd52e9"
Jul 22 16:00:50.549581 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:50.549542    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"]
Jul 22 16:00:50.554282 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:50.554254    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-9911b9-29745600-4gwvs"]
Jul 22 16:00:51.021941 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:51.021903    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c925f699-2319-4e16-9c00-f9e7499790df" path="/var/lib/kubelet/pods/c925f699-2319-4e16-9c00-f9e7499790df/volumes"
Jul 22 16:00:55.182912 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:55.182865    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"]
Jul 22 16:00:55.185767 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:55.185734    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-9911b9-st5rs"]
Jul 22 16:00:57.020872 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:00:57.020831    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05ec3e29-f84b-4b86-a731-466cda0284f5" path="/var/lib/kubelet/pods/05ec3e29-f84b-4b86-a731-466cda0284f5/volumes"
Jul 22 16:01:08.340835 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:08.340744    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79c87dd7d5-qtl5r_8b5239a3-0add-4c40-9a24-9ae1843bc279/manager/0.log"
Jul 22 16:01:08.926803 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:08.926719    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6f74d59b-zr6lg_e5a7131f-df0e-4c1b-9b89-c952ff423059/copy-manifests/0.log"
Jul 22 16:01:08.945183 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:08.945150    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6f74d59b-zr6lg_e5a7131f-df0e-4c1b-9b89-c952ff423059/manager/0.log"
Jul 22 16:01:09.416808 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:09.416775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745570-lfld4_06d9bbb8-8a32-4082-a348-59df96dd159d/cleanup/0.log"
Jul 22 16:01:09.527960 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:09.527914    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745585-8pq52_12614d2e-a2b6-4ba9-8694-26db49afde27/cleanup/0.log"
Jul 22 16:01:09.632507 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:09.632455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745600-dn4gt_e869a481-a2e1-444b-b65f-21873b64d195/cleanup/0.log"
Jul 22 16:01:11.381659 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:11.381636    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x5bdp_e810b721-be3d-4706-a0ac-c525477f9bb2/kuadrant-console-plugin/0.log"
Jul 22 16:01:11.869468 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:11.869425    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qpwhc_74b77eca-c773-43d7-bc37-40b6bb53fa6d/manager/0.log"
Jul 22 16:01:12.572179 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:12.572151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6764d75874-w9pwp_4d117371-97e4-4744-9dcd-47b54f4e8050/kube-auth-proxy/0.log"
Jul 22 16:01:12.815348 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:12.815312    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-nbb58_8916e5f6-7c74-445b-919a-280c95aa3dea/payload-pre-processing/0.log"
Jul 22 16:01:13.513553 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:13.513524    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-789tr_a62971de-6217-4fda-89a9-ad453c3235aa/main/0.log"
Jul 22 16:01:13.968349 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:13.968258    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-4vl84_1a655895-af42-4a51-befb-3c6ba9e8dbd7/main/0.log"
Jul 22 16:01:22.333469 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:22.333430    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4lwsx_5b2ea94f-f08b-41e3-bb42-66b05a6f8e15/global-pull-secret-syncer/0.log"
Jul 22 16:01:22.513277 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:22.513247    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g4wqx_1aa3111a-a405-44fc-96b5-d951e0d8f3ce/konnectivity-agent/0.log"
Jul 22 16:01:22.639618 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:22.639517    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-231.ec2.internal_3132d0a8bfd17a2c358c01f67e7fcefb/haproxy/0.log"
Jul 22 16:01:28.245584 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:28.245554    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x5bdp_e810b721-be3d-4706-a0ac-c525477f9bb2/kuadrant-console-plugin/0.log"
Jul 22 16:01:28.710631 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:28.710588    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qpwhc_74b77eca-c773-43d7-bc37-40b6bb53fa6d/manager/0.log"
Jul 22 16:01:30.324652 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.324554    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/alertmanager/0.log"
Jul 22 16:01:30.351457 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.351424    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/config-reloader/0.log"
Jul 22 16:01:30.373665 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.373633    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/kube-rbac-proxy-web/0.log"
Jul 22 16:01:30.395639 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.395601    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/kube-rbac-proxy/0.log"
Jul 22 16:01:30.418133 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.418109    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/kube-rbac-proxy-metric/0.log"
Jul 22 16:01:30.441891 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.441861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/prom-label-proxy/0.log"
Jul 22 16:01:30.463733 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.463701    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c04afe23-cdfe-48dd-affe-8b4ed48039e9/init-config-reloader/0.log"
Jul 22 16:01:30.546393 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.546356    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-dtb6f_a02f7fd1-174d-47c2-aeaa-062a93041903/kube-state-metrics/0.log"
Jul 22 16:01:30.564312 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.564284    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-dtb6f_a02f7fd1-174d-47c2-aeaa-062a93041903/kube-rbac-proxy-main/0.log"
Jul 22 16:01:30.585275 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.585185    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-dtb6f_a02f7fd1-174d-47c2-aeaa-062a93041903/kube-rbac-proxy-self/0.log"
Jul 22 16:01:30.613375 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.613349    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f789fd5c9-zfch9_ae7a5864-df9b-4d91-b451-5644b081226c/metrics-server/0.log"
Jul 22 16:01:30.650596 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.650562    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-qdnvz_b142f82e-ecc3-47bd-9fa2-9c7e4c9c6ff2/monitoring-plugin/0.log"
Jul 22 16:01:30.685901 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.685860    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2x9qs_1911e184-b292-4084-a23b-048b01b90de7/node-exporter/0.log"
Jul 22 16:01:30.708243 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.708214    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2x9qs_1911e184-b292-4084-a23b-048b01b90de7/kube-rbac-proxy/0.log"
Jul 22 16:01:30.734754 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.734725    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2x9qs_1911e184-b292-4084-a23b-048b01b90de7/init-textfile/0.log"
Jul 22 16:01:30.941475 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.941384    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-56v56_498d107e-12ee-435e-8ca0-5114e13f7e5c/kube-rbac-proxy-main/0.log"
Jul 22 16:01:30.967900 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.967873    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-56v56_498d107e-12ee-435e-8ca0-5114e13f7e5c/kube-rbac-proxy-self/0.log"
Jul 22 16:01:30.990093 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:30.990062    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-56v56_498d107e-12ee-435e-8ca0-5114e13f7e5c/openshift-state-metrics/0.log"
Jul 22 16:01:31.033842 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.033811    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/prometheus/0.log"
Jul 22 16:01:31.058078 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.058052    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/config-reloader/0.log"
Jul 22 16:01:31.083997 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.083968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/thanos-sidecar/0.log"
Jul 22 16:01:31.107065 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.107030    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/kube-rbac-proxy-web/0.log"
Jul 22 16:01:31.130821 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.130788    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/kube-rbac-proxy/0.log"
Jul 22 16:01:31.155108 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.155080    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/kube-rbac-proxy-thanos/0.log"
Jul 22 16:01:31.178555 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.178531    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9940605a-1343-4c4b-b65a-939da48e9032/init-config-reloader/0.log"
Jul 22 16:01:31.210958 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.210871    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-cccgp_4f853648-4345-4232-b7c9-48d74976166c/prometheus-operator/0.log"
Jul 22 16:01:31.228149 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.228124    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-cccgp_4f853648-4345-4232-b7c9-48d74976166c/kube-rbac-proxy/0.log"
Jul 22 16:01:31.251941 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.251915    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-h7h47_1609947f-7031-4245-a946-035973ba3efd/prometheus-operator-admission-webhook/0.log"
Jul 22 16:01:31.356066 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.356037    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b988cc46-6fzsm_a3e3074a-2f4f-47e5-99e9-a5646ea37d05/thanos-query/0.log"
Jul 22 16:01:31.379224 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.379192    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b988cc46-6fzsm_a3e3074a-2f4f-47e5-99e9-a5646ea37d05/kube-rbac-proxy-web/0.log"
Jul 22 16:01:31.402963 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.402931    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b988cc46-6fzsm_a3e3074a-2f4f-47e5-99e9-a5646ea37d05/kube-rbac-proxy/0.log"
Jul 22 16:01:31.426244 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.426214    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b988cc46-6fzsm_a3e3074a-2f4f-47e5-99e9-a5646ea37d05/prom-label-proxy/0.log"
Jul 22 16:01:31.448786 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.448763    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b988cc46-6fzsm_a3e3074a-2f4f-47e5-99e9-a5646ea37d05/kube-rbac-proxy-rules/0.log"
Jul 22 16:01:31.473173 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:31.473089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69b988cc46-6fzsm_a3e3074a-2f4f-47e5-99e9-a5646ea37d05/kube-rbac-proxy-metrics/0.log"
Jul 22 16:01:32.748742 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.748703    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"]
Jul 22 16:01:32.749139 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.749092    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ec3e29-f84b-4b86-a731-466cda0284f5" containerName="revoke-keys"
Jul 22 16:01:32.749139 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.749103    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ec3e29-f84b-4b86-a731-466cda0284f5" containerName="revoke-keys"
Jul 22 16:01:32.749211 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.749191    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05ec3e29-f84b-4b86-a731-466cda0284f5" containerName="revoke-keys"
Jul 22 16:01:32.760733 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.760701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:32.763638 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.763446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zkc2l\"/\"openshift-service-ca.crt\""
Jul 22 16:01:32.763638 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.763527    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zkc2l\"/\"kube-root-ca.crt\""
Jul 22 16:01:32.763638 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.763558    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zkc2l\"/\"default-dockercfg-c4rpr\""
Jul 22 16:01:32.764895 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.764871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"]
Jul 22 16:01:32.908108 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.908069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-sys\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:32.908307 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.908123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-lib-modules\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:32.908307 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.908212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-proc\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:32.908307 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.908255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz6b6\" (UniqueName: \"kubernetes.io/projected/d49af8f2-a9e7-4a31-9ab6-e184483d811c-kube-api-access-nz6b6\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:32.908415 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:32.908308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-podres\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009410 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-podres\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009410 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-sys\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-lib-modules\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-proc\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz6b6\" (UniqueName: \"kubernetes.io/projected/d49af8f2-a9e7-4a31-9ab6-e184483d811c-kube-api-access-nz6b6\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009531    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-podres\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-sys\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-proc\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.009678 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.009601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d49af8f2-a9e7-4a31-9ab6-e184483d811c-lib-modules\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.018030 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.018006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz6b6\" (UniqueName: \"kubernetes.io/projected/d49af8f2-a9e7-4a31-9ab6-e184483d811c-kube-api-access-nz6b6\") pod \"perf-node-gather-daemonset-jbjws\" (UID: \"d49af8f2-a9e7-4a31-9ab6-e184483d811c\") " pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.073259 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.073214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:33.213886 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.213860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"]
Jul 22 16:01:33.216171 ip-10-0-143-231 kubenswrapper[2580]: W0722 16:01:33.216144    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd49af8f2_a9e7_4a31_9ab6_e184483d811c.slice/crio-030dce53be01d3720eb08e7bf6db9c9263c9676531b6c1c007c08d08f66fb7d1 WatchSource:0}: Error finding container 030dce53be01d3720eb08e7bf6db9c9263c9676531b6c1c007c08d08f66fb7d1: Status 404 returned error can't find the container with id 030dce53be01d3720eb08e7bf6db9c9263c9676531b6c1c007c08d08f66fb7d1
Jul 22 16:01:33.833056 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.833031    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b4b876cbd-zbgtb_830ee330-3845-447c-b2f6-2f71b12440b0/console/0.log"
Jul 22 16:01:33.877798 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.877757    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-6s5m6_5d113dfc-e0d3-425e-8f5b-f6a71a6e5aaf/download-server/0.log"
Jul 22 16:01:33.994201 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.994161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws" event={"ID":"d49af8f2-a9e7-4a31-9ab6-e184483d811c","Type":"ContainerStarted","Data":"026f793f562206824c9165f550321acebad65931386069ab9177bde8824f7f95"}
Jul 22 16:01:33.994382 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.994223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws" event={"ID":"d49af8f2-a9e7-4a31-9ab6-e184483d811c","Type":"ContainerStarted","Data":"030dce53be01d3720eb08e7bf6db9c9263c9676531b6c1c007c08d08f66fb7d1"}
Jul 22 16:01:33.994382 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:33.994264    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:34.011882 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:34.011832    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws" podStartSLOduration=2.011816871 podStartE2EDuration="2.011816871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:01:34.010207273 +0000 UTC m=+2903.452290725" watchObservedRunningTime="2026-07-22 16:01:34.011816871 +0000 UTC m=+2903.453900317"
Jul 22 16:01:35.585251 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:35.585222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k4td6_0fe791c4-6fd8-438b-8df7-77722e6eff8a/dns/0.log"
Jul 22 16:01:35.606934 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:35.606897    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k4td6_0fe791c4-6fd8-438b-8df7-77722e6eff8a/kube-rbac-proxy/0.log"
Jul 22 16:01:35.700456 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:35.700431    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hrk9j_c471fe40-e8a7-4c30-bf41-f3bcbe56d275/dns-node-resolver/0.log"
Jul 22 16:01:36.285098 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:36.285064    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7zmpm_1a043e63-87db-4da7-8fbb-0f13536fcc38/node-ca/0.log"
Jul 22 16:01:37.571021 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:37.570985    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6764d75874-w9pwp_4d117371-97e4-4744-9dcd-47b54f4e8050/kube-auth-proxy/0.log"
Jul 22 16:01:37.707208 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:37.707170    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-nbb58_8916e5f6-7c74-445b-919a-280c95aa3dea/payload-pre-processing/0.log"
Jul 22 16:01:38.472653 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:38.472621    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jbrhr_e03051a9-1010-478e-977a-d453bcfa8068/serve-healthcheck-canary/0.log"
Jul 22 16:01:39.192952 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:39.192923    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxqx6_c8093da5-8487-48e9-94bf-ab78aefa884b/kube-rbac-proxy/0.log"
Jul 22 16:01:39.217010 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:39.216980    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxqx6_c8093da5-8487-48e9-94bf-ab78aefa884b/exporter/0.log"
Jul 22 16:01:39.239706 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:39.239676    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxqx6_c8093da5-8487-48e9-94bf-ab78aefa884b/extractor/0.log"
Jul 22 16:01:40.011085 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:40.011058    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zkc2l/perf-node-gather-daemonset-jbjws"
Jul 22 16:01:42.024709 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:42.024602    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79c87dd7d5-qtl5r_8b5239a3-0add-4c40-9a24-9ae1843bc279/manager/0.log"
Jul 22 16:01:42.429326 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:42.429293    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6f74d59b-zr6lg_e5a7131f-df0e-4c1b-9b89-c952ff423059/manager/0.log"
Jul 22 16:01:42.453010 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:42.452924    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c6f74d59b-zr6lg_e5a7131f-df0e-4c1b-9b89-c952ff423059/copy-manifests/0.log"
Jul 22 16:01:43.211719 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:43.211622    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6764d75874-w9pwp_4d117371-97e4-4744-9dcd-47b54f4e8050/kube-auth-proxy/0.log"
Jul 22 16:01:43.347349 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:43.347309    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-nbb58_8916e5f6-7c74-445b-919a-280c95aa3dea/payload-pre-processing/0.log"
Jul 22 16:01:43.799025 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:43.798996    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-947d9d946-hrkk8_ace8cd7f-cffd-4144-b51f-0b65c82b2d21/manager/0.log"
Jul 22 16:01:44.834215 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:44.834186    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6764d75874-w9pwp_4d117371-97e4-4744-9dcd-47b54f4e8050/kube-auth-proxy/0.log"
Jul 22 16:01:44.953398 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:44.953292    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-nbb58_8916e5f6-7c74-445b-919a-280c95aa3dea/payload-pre-processing/0.log"
Jul 22 16:01:50.139226 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:50.139199    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9vpv_0f7f3a35-79f8-491e-82d2-f4cd36f14c4a/kube-multus-additional-cni-plugins/0.log"
Jul 22 16:01:50.160948 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:50.160917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9vpv_0f7f3a35-79f8-491e-82d2-f4cd36f14c4a/egress-router-binary-copy/0.log"
Jul 22 16:01:50.184997 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:50.184968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9vpv_0f7f3a35-79f8-491e-82d2-f4cd36f14c4a/cni-plugins/0.log"
Jul 22 16:01:50.207636 ip-10-0-143-231 kubenswrapper[2580]: I0722 16:01:50.207609    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9vpv_0f7f3a35-79f8-491e-82d2-f4cd36f14c4a/bond-cni-plugin/0.log"